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Introduction 


PASS student attitudes survey data results from the Fall 2011 (baseline or pre-intervention) and Spring 
2014 (third posttest for the Elementary and the Middle School Cohort) administrations are currently 
available and are reported below. There were a total of 14 questions on the PASS that addressed 
classroom achievement in core subjects (reading, math, and science) and different aspects of science 
engagement and science activities in the school. 
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Survey Questions: Student Attitudes Toward Science 


Of the 14 total attitude questions on the student survey, the five questions related to student attitudes 
towards science have been selected for discussion. The data for each of the five questions were 
analyzed separately. To be included in the analysis for a particular question, a student had to meet two 
criteria: 1) a student had to have scores on the multiple choice sections of PASS in both Fall 2011 and 
Spring 2014, and 2) a student had to have answered that question in both Fall 2011 and Spring 2014 to 
ensure we were looking at the same students across the two time points. 


Fall 2011 & Spring 2014 Results: 
Comparison of Phase 1 and Phase 2 Student Attitudes Toward Science 


The first set of analyses examines differences in attitudes toward science between Phase 1 and Phase 2 
students in Fall 2011 and Spring 2014 separately. Of the five items that were selected for discussion, 
three items are Likert-scale items.’ Due to the ordinal (i.e., ranked) nature of the data, the Mann-Whitney 
U test was used to compare the differences between Phase 1 and Phase 2 students’ rankings of each of 
these three survey items. The remaining two items are measured using a nominal scale (i.e., categories 
with no implied ranking).° Therefore, for these two items, the chi-square test of independence was 
employed to evaluate differences in level of agreement between Phase 1 and Phase 2 students. If the 
chi-square test result was statistically significant, standardized residuals were examined to explore which 
cell(s) produced the statistically significant relationship.° 


In addition to the probability level associated with each statistical outcome, in the tables following, an 
effect size is provided as an indicator of the impact or “practical” significance of the treatment (i.e., being 
a Phase 1 student). The “effect size” is a descriptive statistic that indicates the magnitude of the 
difference (in standard deviation units) between two measures. Except in the case of the chi-square test, 
for the current between-group comparison study, a positive effect size would indicate a higher outcome 
for Phase 1 students, while a negative effect size would indicate a higher outcome for the non-treatment 
group (Phase 2). For the chi-square test, the effect size is a measure of the strength of association 
between the group and the outcome, with a larger effect size indicating a stronger relationship between 
group membership and the outcome. Based on guidelines from the What Works Clearinghouse (WWC), 
part of the research arm of the U.S. Department of Education, a Hedge’s g effect size of at least 0.25 is 
considered “substantively important” (What Works Clearinghouse, 2014). In cases where the default 


effect size for a statistical test was not calculated as the Hedge’s g, (e.g., the phi coefficient (¢) for a chi- 


square test), the default effect size was converted to Hedge’s g for interpretation. As the analyses were 
considered exploratory in nature, no correction for multiple comparisons was made. 


* The item “Do you like science” was measured using a 3-point Likert scale (0 = Not At All; 1 = A Little; 2 = A Lot). The items “How 
often do you talk to your family about what you do in science class?” and “How often do you talk to your friends about what you do in 
science class?” were also measured using a 3-point Likert scale (0 = Never or Almost Never; 1 = Some; 2 = A Lot). 

? The items “Do you think science will be useful when you are older?” and “Would you like to be a scientist when you are older?” 
were measured using a nominal scale (0 = No; 1 = Maybe; 2 = Yes). 

3 In our analyses, the critical value for a standardized residual is -1.96 and +1.96, which corresponds to a level of significance of 
0.05. 
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Comparison of Student Attitudes Toward Science 
from Fall 2011 to Spring 2014 


In addition to evaluating attitudinal differences between Phase 1 and Phase 2 students in Fall 2011 and 
Spring 2014 separately, a second set of analyses was conducted to assess how students’ attitudes 
toward science changed from Fall 2011 to Spring 2014 within the Phase 1 and Phase 2 groups. Again, 
because of the non-interval, longitudinal, and matched-pair nature of the data, nonparametric methods for 
related samples were employed to compare differences between the two time points for the Phase 1 and 
Phase 2 groups separately. Specifically, the Wilcoxon Signed-rank test was performed on Likert-scale 
items and the marginal homogeneity test was performed on nominal scale items. The effect size measure 
r for the Wilcoxon signed-rank test was calculated.“ In the case of the present within-group comparison 
study, a positive effect size would indicate a higher outcome in Spring 2014, while a negative effect size 
would indicate a higher outcome in Fall 2011. No effect size measure is available for the marginal 
homogeneity test. 


Results for All Regions combined are presented first, followed by the outcomes for the Houston Independent 
School District (HISD), the New Mexico region, and the North Carolina region. A summary of the Key Findings 
for each set of analyses is presented at the beginning of each report, followed by information on the samples 
included, the detailed outcomes by grade level (i.e., elementary cohort and middle school cohort), and either 
between or within the Phase 1 and Phase 2 groups. 


“ The calculation of the effect size of Wilcoxon Signed-rank test is r = — where N is the total of number of observations (See 


http://yatani.jp/HClstats/WilcoxonSigned). The effect was negligible if |r| < 0.10, small if |r| > 0.10, medium if |r| > 0.30, and large if 
|r| > 0.50. 


Summative Report Section 5: Student Attitudes 5 


All Regions: 
Results for Spring 2014 PASS 
Student Attitudes Toward Science 
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All Regions 
Spring 2014 PASS Student Attitudes 
Key Findings for Phase 1 


For students across all three regions, the following outcomes favoring Phase 1 students were found on 
the Spring 2014 PASS Student Attitudes section. 


e For the between-group differences, on the nominal question “Do you think science will be useful 
to you when you are older?”, a higher percentage of Phase 1 students in the Elementary Cohort 
responded “Yes” in Spring 2014 (51.7% vs. 49.0%). However, while this difference was 
statistically significant, it was not substantively important. For the Likert-scale item “How often do 
you talk to your friends about what you do in science class?”, Phase 1 students in the Elementary 
Cohort had statistically higher scores in Spring 2014 (i.e. were more likely to talk to their friends), 
but the associated effect size was not substantively important. 
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All Regions: 
Fall 2011 to Spring 2014 PASS Student Attitudes Results 


PASS student attitudes survey data results across all three regions from the Fall 2011 (baseline or pre- 


intervention) and Spring 2014 (third posttest for the Elementary and the Middle School Cohort) 
administrations are currently available and are reported below. 


Survey Questions: Student Attitudes Toward Science 


Of the 14 total attitude questions on the student survey, the five questions related to student attitudes 


towards science have been selected for discussion. See Table A - 1 and Table A - 2 in Appendix A for the 


outcomes on all 14 student attitudes questions. Table 1 and Table 2 show the final analytic sample sizes 
included for the Elementary and Middle School cohorts respectively”. 


Table 1. PASS, All Regions, Spring 2014: Samples for the Survey Analyses for Elementary Cohort 
Sample Phase 1 Phase 2 


Initial Samples? 

| like science. 2,064 1,558 
How often do you talk to your family about what you do in science class? 2,067 1,559 
How often do you talk to your friends about what you do in science class? 2,038 1,552 
Do you think science will be useful when you are older? 2,099 1,585 
Would you like to be a scientist when you are older? 2,068 1,551 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 


Table 2. PASS, All Regions, Spring 2014: Samples for the Survey Analyses for Middle School Cohort 
Sample Phase 1 Phase 2 


Initial Samples? 

| like science. 883 983 
How often do you talk to your family about what you do in science class? 889 1,007 
How often do you talk to your friends about what you do in science class? 887 1,000 
Do you think science will be useful when you are older? 894 1,011 
Would you like to be a scientist when you are older? 880 1,003 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 


5 The sample size for the Elementary cohort across all three regions includes the students from New Mexico, North Carolina and HISD, 
whereas the sample size for the Middle School cohort includes only students from New Mexico and North Carolina because HISD Middle 
School students did not take Fall 2011 PASS. 
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Fall 2011 & Spring 2014 All Regions Results: 
Comparison of Phase 1 and Phase 2 Student Attitudes Toward Science 


The results of the Mann-Whitney U test for the Elementary Cohort and Middle School cohort are 
summarized in Table 3 and Table 5, respectively. The results of the chi-square test of independence for 
the Elementary Cohort and Middle School cohort are summarized in Table 4 and Table 6, respectively. 


Elementary Cohort Results (Figure 1, page 17): Between Group Comparisons 


» “I like science” 


As shown in Table 3, the Mann-Whitney U test results revealed that Phase 1 students had slightly higher 
scores (i.e., a higher level of agreement) than Phase 2 students in both Fall 2011 and Spring 2014. 
However, the differences were neither statistically significant nor substantively important. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 3, the Mann-Whitney U test results revealed that Phase 1 students had slightly higher 
scores (i.e., a higher level of agreement) than Phase 2 students in both Fall 2011 and Spring 2014. 
However, the differences were neither statistically significant nor substantively important. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 3, the Mann-Whitney U test results revealed that Phase 1 students had higher scores 
(i.e., a higher level of frequency ) than Phase 2 students in both Fall 2011 and Spring 2014, with the 
difference in Spring 2014 being statistically significant (Z = -3.78, p < 0.001, g = 0.13). However, neither 
of the associated effect sizes was substantively important. 


=" “Do you think science will be useful when you are older?” 


As shown in the top section of Table 4, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was not statistically significantly different in Fall 
2011, whereas the difference was statistically significant in Spring 2014 (y°(2) = 7.54, p = 0.023, g = 0.09). 
However, neither difference was substantively important. Examination of the cell standardized residuals 
(s.r.) showed that none of the cells individually was a major contributor to the overall statistically 
significant relationship between Phase and response for students in Spring 2014. Rather, they worked 
jointly to contribute to the overall statistical significance, with a higher percentage of Phase 1 students 
responding “Yes” (51.7% vs. 49.0%). 


=» “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 4, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was not statistically significantly 
different in Spring 2014, whereas the difference was statistically significant in Fall 2011 (77(2) = 9.28, p = 
0.010, g = 0.10). However, neither difference was substantively important. Examination of the cell 
standardized residuals (s.r.) showed that none of the cells individually was a major contributor to the 
overall statistically significant relationship between Phase and response for students in Fall 2011. Rather, 
they worked jointly to contribute to the overall statistical significance, with a higher percentage of Phase 1 
students responding “Yes” (25.1% vs. 20.8%). It should be noted that the largest percentage of both 
Phase 1 and Phase 2 students at both time points responded “No”, with the percentage for Phase 1 being 
smaller at both time points. 
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Table 3. Results of Mann-Whitney U Test: Comparison of Overall Elementary Cohort Phase 1 and Phase 2 Students’ Responses 
Fall 2011 Spring 2014 


Phase 1 Phase 2 Phase 1 Phase 2 
N Mean N Mean Z g N Mean N Mean Z g 
| like science. 2,064 162 1558 160 -080 0.03 | 2,064 153 1,558 1.50 -1.79 0.06 


How often do you talk to 
your family about what 2,067 104 1559 100 -191 0.06 | 2,067 091 1,559 0.89 -1.27 0.04 
you do in science class? 
How often do you talk to 
your friends about what 2,038 O77 1,552 073 -162 0.05 | 2,038 073 1,552 0.64 -3.78"* 0.13 
you do in science class? 
*** 0 < 0.001, two-tailed. A negative g indicates that Phase 2 had a higher score relative to Phase 1. 


Table 4. Results of Chi-Square Test: Comparison of Overall Elementary Cohort Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 


Fall 2011 Spring 2014 
(y?(2) = 5.73, p = 0.057, @ = 0.04, g = 0.08) (v?(2) = 7.54, p = 0.023*, @ = 0.05, g = 0.09) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI n % SI. n % SI n % Sw 
Yes 1,077 51.3 0.9 761 48.0 -1.1 1,086 51.7 0.8 776 49.0 -0.9 
Maybe 881 42.0 -0.5 692 43.7 0.6 888 42.3 -1.2 735 46.4 1.4 
No 141 6.7 -1.2 132 8.3 1.3 125 6.0 11 74 4.7 -1.3 


Would you like to be a scientist when you are older? 


Fall 2011 Spring 2014 
(v2(2) = 9.28, p = 0.010*, @ = 0.05, g = 0.10) (v?(2) = 4.34, p = 0.114, d = 0.04, g = 0.07) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % Sw. n % SL n % SI n % SI 
Yes 520 25.1 17 323 20.8 -2.0 207 10.0 0.9 133 8.6 -1.1 
Maybe 640 30.9 -0.7 511 33.0 0.8 695 33.6 0.6 494 31.9 -0.7 
No 908 43.9 -0.7 717 46.2 0.8 1,166 56.4 -0.8 924 59.6 0.9 


Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 
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Middle School Cohort Results (Figure 2, page 19): Between Group Comparisons 
» “I like science” 


As shown in Table 5, the Mann-Whitney U test results revealed that although Phase 1 students had 
higher scores than Phase 2 students in Fall 2011, the difference was neither statistically significant nor 
substantively meaningful. In contrast, compared to Phase 1 students, Phase 2 students scored higher in 
Spring 2014. However, the magnitude of the difference was neither statistically significant nor 
substantively important. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 5, the Mann-Whitney U test results revealed that although Phase 2 students had 
slightly higher scores than Phase 1 students in Fall 2011,the difference was neither statistically significant 
nor substantively important. In contrast, compared to Phase 1 students, Phase 2 students had statistically 
significantly higher scores in Spring 2014 (Z = 3.52, p < 0.001, g = -0.16), but again, the associated effect 
size was not substantively important. 


=" “How often do you talk to your friends about what you do in science class?” 


As shown in Table 5, Phase 2 students had slightly higher scores than Phase 1 students in both Fall 2011 
and Spring 2014, with the difference in Spring 2014 being statistically significant (Z = 2.15, p = 0.031, 
g = -0.10). However, neither of the associated effect sizes was substantively important. 


* “Do you think science will be useful when you are older?” 


As shown in the top section of Table 6, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was not statistically significant in Fall 2011, 
whereas the difference was statistically significant in Spring 2014 (y7(2) = 25.59, p < 0.001, g = 0.23). 
However, neither difference was substantively important. Examination of the cell standardized residuals 
(s.r.) showed that the responses of “Yes” and “No” by both groups were major contributors to the overall 
statistically significant relationship between Phase and students’ response in Spring 2014, with a larger 
than expected percentage of Phase 2 students responding “Yes” in Spring 2014 (44.1% vs. 34.9%), and 
a larger than expected percentage of Phase 1 students responding “No” (14.2% vs. 8.4%). Note that the 
largest percentage of both Phase 1 and Phase 2 students responded “Yes” in Fall 2011, but “Maybe” in 
Spring 2014. 


=» “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 6, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was neither statistically significant 
nor substantively important in either Fall 2011 or Spring 2014. The largest percentage of Phase 1 and 
Phase 2 students at both time points responded “No”. 
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Table 5. Results of Mann-Whitney U Test: Comparison of Overall Middle School Cohort Phase 1 and Phase 2 Students’ Responses 
Fall 2011 Spring 2014 


Phase 1 Phase 2 Phase 1 Phase 2 
N Mean N Mean Z g N Mean N Mean Z g 
| like science. 883 1.49 983 1.47 0.71 0.03 883 1.15 983 1.21 1.78 -0.08 


How often do you 
talk to your family 
about what you do 
in science class? 
How often do you 
talk to your friends 
about what you do 
in science class? 
* p < 0.05, two-tailed; *** p < 0.001, two-tailed. A negative g indicates that Phase 2 had a higher score relative to Phase 1. 


889 0.97 1,007 0.98 0.41 ~— -0.02 889 0.55 1,007 0.64 3.52** = -0.16 


887 0.73 1,000 0.78 117 ~~ -0.05 887 0.62 1,000 0.68 2.15* -0.10 


Table 6. Results of Chi-Square Test: Comparison of Overall Middle School Cohort Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 


Fall 2011 Spring 2014 
(v2 (2) = 1.30, p = 0.523, @ = 0.03, g = 0.05) (v2 (2) = 25.59, p <0.001*, @ = 0.12, g = 0.23) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % Sw. n % SI n % SI n % SI. 
Yes 493 55.2 -0.2 565 55.9 0.1 312 34.9 -2.3 446 44.1 2.2 
Maybe 374 41.8 0.4 407 40.3 -0.4 455 50.9 0.8 480 47.5 -0.7 
No 27 3.0 -0.7 39 3.9 0.7 127 14.2 2.8 85 8.4 -2.6 


Would you like to be a scientist when you are older? 


Fall 2011 Spring 2014 
(v2(2) = 5.84, p = 0.05, @ = 0.06, g = 0.11) (v2(2) = 0.33, p = 0.846, @ = 0.01, g = 0.03) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI n % SI n % SI. n % SI 
Yes 81 9.2 1.4 67 6.7 -1.3 36 4.1 0.4 36 3.6 -0.4 
Maybe 306 34.8 0.5 330 32.9 -0.5 204 23.2 -0.1 236 23.5 0.1 
No 493 56.0 -0.9 606 60.4 0.9 640 72.7 0.0 731 72.9 0.0 


Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 
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All Regions Results: 
Comparison of Student Attitudes Toward Science 
from Fall 2011 to Spring 2014 


The results of the Wilcoxon Signed-rank test for the Elementary and Middle School cohorts are 
summarized in Table 7 and Table 9, respectively. The results of the marginal homogeneity test for the 
Elementary and Middle School cohorts are summarized in Table 8 and Table 10, respectively. 


Elementary Cohort Results (Figure 1, page 17): Within Group Comparisons 
=» “I like science” 


As shown in Table 7, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 5.34, p < 0.001, r= -0.08 
and Z = 5.35, p < 0.001, r = -0.10, respectively). However, the associated effects were negligible for 
Phase 1 and small for Phase 2. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 7, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 6.73, p < 0.001, r= -0.10 
and Z = 5.09, p < 0.001, r = -0.09, respectively). However, the associated effects were negligible. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 7, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 2.28, p = 0.022, r= -0.04 
and Z = 3.92, p < 0.001, r = -0.07, respectively). However, the associated effects were negligible. 


* “Do you think science will be useful when you are older?” 


As shown in Table 8, the marginal homogeneity tests showed that there was a statistically significant 
difference in the level of agreement for this question within the Phase 2 student group from Fall 2011 to 
Spring 2014 (MH = -2.23, p = 0.001), with the level of agreement being higher in Spring 2014, whereas 
the difference within the Phase 1 student group, while higher in Spring 2014, was not statistically 
significant. 


» “Would you like to be a scientist when you are older?” 


As shown in Table 8, the marginal homogeneity tests showed that there were statistically significant 
differences between Fall 2011 and Spring 2014 in the level of agreement for this question within both the 
Phase 1 and Phase 2 groups (MH = 13.01, p < 0.001 and MH = 10.83, p < 0.001, respectively), with the 
level of agreement being higher in Fall 2011 for both groups. 
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Table 7. Results of Wilcoxon Signed-Rank Test: Comparison of Overall Elementary Cohort 


Students’ Responses within the Phase 1 and Phase 2 Groups 
Phase 1 Phase 2 | 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 

N Mean N Mean Z r N Mean N Mean Z r 
| like science. 2,064 1.62 2,064 1.53 5.34** = -0.08 | 1,558 1.60 1,558 1.50 5.35*** — -0.10 
How often do you talk 
to your family about 2,067 1.04 2067 091 6.73%* -010 | 1,559 1.00 1,559 0.89 5.09%*  -0.09 
what you do in science 
class? 
How often do you talk 
toyour friends about = 53g 0.77 «2,088-«0.73~=S2.28* «0.04 | 1552 0.73 «1,552 064 3.92%" 0.07 
what you do in science 
class? 


** 9 < 0.01, two-tailed; *** p < 0.001, two-tailed. A negative r indicates that Fall 2011 had a higher score relative to Spring 2014. 


Table 8. Results of Marginal Homogeneity Test: Comparison of the Distributions of Overall Elementary Cohort 


Students’ Responses within the Phase 1 and Phase 2 Grou 


Marginal Homogeneity Test 


Do you think science will be useful to you when 


you are older? 


Would you like to be a scientist when you are 


older? 


N Standard MH Statistic 
Phase 1 2,099 -0.68 0.498 
Phase 2 1,585 -2.23 0.026* 
Phase 1 2,068 13.01 <0.001* 
Phase 2 1,551 10.83 <0.001* 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative to 


Fall 2011. 
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Middle School Cohort Results (Figure 2, page 19): Within Group Comparisons 
» “I like science” 


As shown in Table 9, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 12.47, p < 0.001, r=- 
0.30 and Z = 11.10, p < 0.001, r = -0.25, respectively). The associated effect size for both group 
differences was small. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 9, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 14.24, p < 0.001, r=- 
0.34 and Z = 13.51, p < 0.001, r = -0.30, respectively). The associated effects for both groups were 
medium. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 9, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 4.20, p < 0.001, r=-0.10 
and Z = 3.45, p = 0.001, r= -0.08, respectively). The associated effects for both groups were small. 


*" “Do you think science will be useful when you are older?” 


As shown in Table 10, the marginal homogeneity tests showed that there were statistically significant 
differences between Fall 2011 and Spring 2014 in the level of agreement for this question within both the 
Phase 1 and Phase 2 groups (MH = 11.26, p < 0.001 and MH = 6.92, p < 0.001, respectively), with the 
level of agreement being higher in Fall 2011 for both groups. 


=» “Would you like to be a scientist when you are older?” 


As shown in Table 10, the marginal homogeneity tests showed that there were statistically significant 
differences between Fall 2011 and Spring 2014 in the level of agreement for this question within both the 
Phase 1 and Phase 2 groups (MH = 9.09, p < 0.001 and MH = 7.08, p < 0.001, respectively), with the 
level of agreement being higher in Fall 2011 for both groups. 
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Table 9. Results of Wilcoxon Signed-Rank Test: Comparison of Overall Middle School Cohort Students’ Responses 


within the Phase 1 and Phase 2 Groups 
Phase 1 Phase 2 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 
N Mean N Mean Z r N Mean N Mean Z r 
| like science. 883 1.49 883 1.15 12.47*** — -0.30 983 1.47 983 1.21 11.10% = -0,25 


How often do you talk 
to your family about 
what you do in 
science class? 

How often do you talk 
to your friends about 
what you do in 
science class? 

*** 0 < 0.001, two-tailed. A negative r indicates that Fall 2011 had a higher score relative to Spring 2014 


889 0.97 889 055 14.24** -0.34 | 1,007 0.98 1,007 064 13.51** -0.30 


887 = 0.73, 887 ~— (0.62 4.20"* -0.10 | 1000 0.78 1,000 0.68 3.45** —-0.08 


Table 10. Results of Marginal Homogeneity Test: Comparison of the Distributions of Overall Middle School Cohort Students’ 
Responses within the Phase 1 and Phase 2 Groups 


Marginal Homogeneity Test 


N Standard MH Statistic 
Do you think science will be useful to you when you Phase 1 894 11.26 <0.001' 
ile olen? Phase 2 1,011 6.92 <0.001° 
Phase 1 880 9.09 <0.001" 
Would you like to be a scientist when you are older? 
Phase 2 1,003 7.08 <0.001" 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative 
to Fall 2011. 
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Figure 1. Student Attitude Survey, All Regions: Fall 2011 and Spring 2014: Elementary Cohort Question Responses by Group 


I like Science ... 
Not at all A little A lot 
Phase 1 27.7% 
Fall 2011 


Phase 2 ER liza 


Not at all A little A lot 
Phase 1 EE 38.2% 
Spring 2014 


Phase 2 ER a1.a% 


How often do you talk to your family about what you do in science class? 


Never or almost never Some Alot 
Phase 1 49.7% 
Fall 2011 


ehne? 48.5% 


Never or almost never Some Alot 
Phase 1 61.2% 
Spring 2014 


Phase 2 ca. 
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How often do you talk to your friends about what you do in science class? 


Never or almost never Some Alot 
Phase 1 37.4% 
Fall 2011 


Phase 2 44.4% 38.3% 


Never or almost never Some Alot 
Phase 1 39.1% 49.2% 
Spring 2014 


Phase 42.3% 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase 4 42.0% 
Fall 2011 


Phase 2 43.7% 


No Maybe Yes 
Phase 1 EXT 42.3% 
Spring 2014 


Phase 2 46.4% 


Would you like to be a scientist when you are older? 


No Maybe Yes 
Phase 1 43.9% 30.9% 
Fall 2011 


No Maybe Yes 
Phase 1 56.4% 33.6% 
Spring 2014 
Phase 2 59.6% 31.9% 


Summative Report Section 5: Student Attitudes 


18 


Figure 2. Student Attitude Survey, All Regions: Fall 2011 and Spring 2014: Middle School Cohort Question Responses by Group 


I like Science ... 
Not at all A little A lot 
Phase 1 Ef 44.1% 
Fall 2011 


Phase 2 EE 44.0% 


Not at all A little Alot 
Phase 60.4% 
Spring 2014 


Phase 2 EXT 61.4% 


How often do you talk to your family about what you do in science class? 


Never or almost never Some Alot 
Phase 1 59.3% 
Fall 2011 


Phase 2 PRET? 62.3% 


Never or almost never Some Alot 
Phase 1 45.3% 0 
Spring 2014 
Phase 2 40.7% 54.9% La%4 
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How often do you talk to your friends about what you do in science class? 


Never or almost never Some Alot 
Phase 1 37.9% 51.2% 
Fall 2011 


Phase 2 47.3% 


Never or almost never Some Alot 
Phase 1 47.4% 
Spring 2014 


Phase 2 49.9% 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase 1 Ej 41.8% 
Fall 2011 


enese? a 40.3% 


No Maybe Yes 
Phase 1 SETEZZ 50.9% 
Spring 2014 


Phase 2 ERD 475% 


Would you like to be a scientist when you are older? 


No Maybe Yes 
Phase 1 56.0% 34.8% 
Fall 2011 
Phase 2 60.4% 32.9% 
No Maybe Yes 
Phase 1 72.7% 23.2% 1% 
Spring 2014 
Phase 2 72.9% 23.5% 
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Overall Summary of PASS Student Attitudes Results 


Between-qroup difference 


For the three Likert-scale items (‘I like science”, “How often do you talk to your family about what you do 
in science class?”, and “How often do you talk to your friends about what you do in science class?”), for 
both the Elementary and Middle School cohort students across all three regions, while there were some 
statistically significant findings in Spring 2014, none of the three items examined exhibited meaningful 
differences between Phase 1 and Phase 2 students in either Fall 2011 or Spring 2014. In other words, 
Phase 1 and Phase 2 students in both cohorts essentially had similar attitudes toward science and talked 
to their families and friends about science with similar frequencies in both Fall 2011 and Spring 2014. 


For the two nominal scale items (“Do you think science will be useful to you when you are older?” and 
‘Would you like to be a scientist when you are older?”), there were statistically significant differences 
between Phase 1 and Phase 2 students in both cohorts on (1) the first question in Spring 2014, with 
Phase 1 students in the Elementary Cohort having a higher percentage of responding “Yes” (51.7% vs. 
49.0%) while Phase 2 students in the Middle School Cohort had a higher percentage of responding “Yes” 
(44.1% vs. 34.9%), and (2) on the second question for the Elementary Cohort in Fall 2011, with Phase 1 
students having a higher percentage of responding “Yes” (25.1% vs. 20.8%). However, none of the 
differences were substantively meaningful. In other words, Phase 1 and Phase 2 students in both cohorts 
essentially had similar attitudes towards these two nominal questions in both Fall 2011 and Spring 2014. 


Within-group difference 


When looking at results within the Phase 1 and Phase 2 groups, for the three Likert-scale items (‘I like 
science”, “How often do you talk to your family about what you do in science class?”, and “How often do 
you talk to your friends about what you do in science class?”), Phase 1 and Phase 2 students in both the 
Elementary and Middle School Cohorts liked science more or were more likely to talk to their friends or 
families about science in Fall 2011 compared to Spring 2014. All differences were statistically significant, 
with the largest magnitude being medium for the item “How often do you talk to your family about what 
you do in science class?” within both the Phase 1 and Phase 2 groups in the Middle School Cohort. 


For the two nominal scale items (“Do you think science will be useful to you when you are older?” and 
‘Would you like to be a scientist when you are older?”), there were statistically significant differences in 
the level of agreement within both groups from Fall 2011 to Spring 2014 for both the Elementary and 
Middle School Cohorts except for the responses of Phase 1 students in the Elementary Cohort to the 
question “Do you think science will be useful to you when you are older?”. Only one of the statistically 
significant differences, for the Phase 2 students in the Elementary Cohort on the question “Do you think 
science will be useful to you when you are older?” had a higher percentage of agreement in Spring 2014. 
All of the remaining statistically significant differences demonstrated higher levels of agreement in Fall 
2011. However, since no effect size measure for the marginal homogeneity test was available, we do not 
know whether or not these statistically significant differences were substantively important. 
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Houston Independent School District: 
Results for Spring 2014 PASS 
Student Attitudes Toward Science 
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Houston Independent School District (HISD) 
Spring 2014 PASS Student Attitudes 
Key Findings for Phase 1 


For all students combined in HISD, the following outcomes favoring Phase 1 Elementary Cohort students 
were found on the Spring 2014 PASS Student Attitudes section. 


e For the between-group differences, where there was baseline equivalence between Phase 1 and 
Phase 2 students, there was a statistically significant difference between groups in Spring 2014 
on the question “Do you think science will be useful to you when you are older?”, with the largest 
percentage of Phase 1 students responding “Yes” in both Fall 2011 and Spring 2014, whereas 
the largest percentage of Phase 2 students responded “Yes” in Fall 2011 and “Maybe” in Spring 
2014, but the difference was not substantively important. In Spring 2014, over half of Phase 1 
students (54.4%) responded “Yes” compared to less than half (48.8%) of Phase 2 students. 


e For the within-group differences, neither the Phase1 nor Phase 2 Elementary or Middle School 
cohorts had statistically significantly better outcomes in Spring 2014 vs. the baseline (Fall 2011 or 
Spring 2012 respectively). 
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Houston: 
Fall 2011 or Spring 2012 to Spring 2014 PASS Student Attitudes Results 


PASS student attitudes survey data results in HISD from the Fall 2011 (baseline or pre-intervention for 
the Elementary Cohort) or Spring 2012° (baseline or pre-intervention for the Middle School Cohort) and 
Spring 2014 (third posttest for the Elementary Cohort and the second posttest for the Middle School 
Cohort) administrations are currently available and are reported below. 


Survey Questions: Student Attitudes Toward Science 


Of the 14 total attitude questions on the student survey, the five questions related to student attitudes 
towards science have been selected for discussion. See Table A - 3and Table A - 4 in Appendix A for the 
outcomes on all 14 student attitudes questions. Table 11 and Table 12 show the final analytic HISD 
sample sizes included for the Elementary and Middle School cohorts respectively. 


Table 11. PASS, Houston, Spring 2014: Samples for the Survey Analyses for Elementary Cohort 
Item Phase 1 Phase 2 


Initial Samples? 801 614 
| like science. 561 407 
How often do you talk to your family about what you do in science class? 556 409 
How often do you talk to your friends about what you do in science class? 557 405 
Do you think science will be useful when you are older? 568 418 
Would you like to be a scientist when you are older? 561 406 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 


Table 12. PASS, Houston, Spring 2014: Samples for the Survey Analyses for Middle School Cohort 
Item Phase 1 Phase 2 


Initial Samples? 179 121 
| like science. 146 97 
How often do you talk to your family about what you do in science class? 151 99 
How often do you talk to your friends about what you do in science class? 149 99 
Do you think science will be useful when you are older? 150 100 
Would you like to be a scientist when you are older? 149 98 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 


® The Middle School Cohort in HISD did not take the PASS in the Fall 2011, so Spring 2012 was chosen as the baseline for the 
Middle School Cohort. 
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Fall 2011 (or Spring 2012) & Spring 2014 Houston Results: 
Comparison of Phase 1 and Phase 2 Student Attitudes Toward Science 


The results of the Mann-Whitney U test for Elementary Cohort and Middle School Cohort are summarized 
in Table 13 and Table 15, respectively. The results of the chi-square test of independence for the 
Elementary and Middle School cohorts are summarized in Table 14 and Table 16 respectively. 


Elementary Cohort Results (Figure 3, page 37): Between Group Comparisons 


» “I like science” 


As shown in Table 13, Phase 1 students had slightly higher scores (i.e., a higher level of agreement) than 
Phase 2 students in Spring 2014, whereas Phase 2 students had slightly higher scores in Fall 2011. 
However, the Mann-Whitney U test results revealed such differences were neither statistically significantly 
nor substantively importantly different in either Fall 2011 or Spring 2014. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 13, Phase 1 students had slightly higher scores (i.e., a higher frequency) than Phase 2 
students in both Fall 2011 and Spring 2014. However, the Mann-Whitney U test results revealed such 
differences were neither statistically significantly nor substantively importantly different in either Fall 2011 
or Spring 2014. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 13, Phase 1 students had slightly higher scores (i.e., a higher frequency) than Phase 2 
students in Spring 2014, whereas Phase 2 students had higher scores in Fall 2011. However, the Mann- 
Whitney U test results revealed such differences were neither statistically significantly nor substantively 
importantly different in either Fall 2011 or Spring 2014. 


=" “Do you think science will be useful when you are older?” 


As shown in the top section of Table 14, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was neither statistically significantly nor 
substantively importantly different in Fall 2011. In contrast, the level of agreement between two groups 
was Statistically significantly different in Spring 2014, although the difference was not substantively 
important. Note that the largest percentage of Phase 1 students responded “Yes” in both Fall 2011 and 
Spring 2014, whereas the largest percentage of Phase 2 students responded “Yes” in Fall 2011 and 
“Maybe” in Spring 2014. 


=» “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 14, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was neither statistically significantly 
nor substantively importantly different in either Fall 2011 or Spring 2014. For both Phase 1 and Phase 2 
students in Fall 2011, the percentage of students selecting each response category responses was 
almost the same, whereas in Spring 2014, the largest percentage of both Phase 1 and Phase 2 students 
responded “No” (49.6% for Phase 1 students and 51.5% for Phase 2 students). 
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Table 13. Results of Mann-Whitney U Test: Comparison of Houston Elementary Cohort Phase 1 and Phase 2 Students’ Responses 
Fall 2011 Spring 2014 


Phase 1 Phase 2 Phase 1 Phase 2 
N Mean N Mean Z g N Mean N Mean Z g 
| like science. 561 1.68 407 1.71 0.86  -0.06 561 1.55 407 154 -073 0.05 


How often do you talk to 
your family about what you 556 1.13 409 111 = -0.31 0.02 556 0.91 409 0.88 -0.70 0.05 
do in science class? 
How often do you talk to 
your friends about what 557 ~— 0.93 405 0.99 121 -0.08 557 0.76 405 0.69 -1.22 0.08 
you do in science class? 

Note. A negative g indicates that Phase 2 has a higher score relative to Phase 1. 


Table 14. Results of Chi-Square Test: Com 


parison of Houston Elementary Cohort Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 


Fall 2011 Spring 2014 
(X2(2) = 3.61, p = 0.165, Cramer's V = 0.06, g = 0.12) (X2(2) = 10.14, p = 0.006*, Cramer's V = 0.10, g = 0.20) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI n % Sh n % SI n % Sw 
Yes 289 50.9 -0.8 238 56.9 1.0 309 54.4 0.9 202 48.3 -1.0 
Maybe 247 43.5 0.9 158 37.8 = -1.0 227 40.0 -1.4 204 48.8 1.6 
No 32 5.6 0.2 22 5.3 -0.2 32 5.6 13 12 2.9 -1.5 


Would you like to be a scientist when you are older? 


Fall 2011 Spring 2014 
(X2(2) = 1.08, p = 0.584, Cramer's V = 0.03, g = 0.07) (X2(2) = 0.36, p = 0.834, Cramer's V = 0.02, g = 0.04) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % Sf. n % Sif. n % Si. n % Si. 
Yes 184 32.8 0.1 131 323 -01 59 10.5 0.1 42 10.3 = -0.1 
Maybe 189 33.7 -0.5 149 36.7 = 0.6 224 39.9 0.3 155 38.2 = -0.3 
No 188 33.5 0.4 126 310 -05 278 49.6 -0.3 209 51.5 0.3 


Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 
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Middle School Cohort Results (Figure 4, page 39): Between Group Comparisons 


» “I like science” 


As shown in Table 15, the Mann-Whitney U test results revealed that Phase 2 students had statistically 
significantly and substantively higher scores (i.e., a higher level of agreement) than Phase 1 students in 
both Spring 2012 and Spring 2014 (Z = 2.80, p = 0.005, g = -0.37 and Z = 4.23, p < 0.001, g = -0.55, 
respectively). 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 15, the Mann-Whitney U test results revealed that Phase 2 students had higher scores 
(i.e., a higher frequency) than Phase 1 students in both Spring 2012 and Spring 2014, but only the 
difference in Spring 2014 was statistically significant and substantively important (Z = 3.33, p = 0.001, g = 
-0.43). 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 15, the Mann-Whitney U test results revealed that Phase 2 students had statistically 
significantly and substantively higher scores (i.e., a higher frequency) than Phase 1 students in both 
Spring 2012 and Spring 2014 (Z = 2.87, p = 0.004, g = -0.37 and Z = 2.13, p = 0.034, g = -0.28, 
respectively). 


*" “Do you think science will be useful when you are older?” 


As shown in the top section of Table 16, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was both statistically significantly and 
substantively importantly different in Spring 2014 (y7(2) = 6.62, p = 0.036, g = 0.32), whereas the level of 
agreement between two groups in Spring 2012 was only substantively importantly different. Examination 
of the cell standardized residuals (s.r.) showed that none of the cells individually was a major contributor 
to the overall statistically significant relationship between Phase and response for students at either time 
point. Rather, they worked jointly to contribute to the overall statistical significance, with both groups 
having the largest percentage of respondents give the same response at each time point, but a higher 
percentage of Phase 2 students responding “Yes” in Spring 2012 (61.0% vs. 54.0%) and “Maybe” in 
Spring 2014 (52.0% vs. 49.3%). Phase 2 (42.0%) also had a larger percentage of students respond “Yes” 
in Spring 2014 compared to Phase 1 (34.0%). 


=» “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 16, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was statistically significantly different 
in Spring 2012 (77(2) = 10.69, p = 0.005, g = 0.41), but not in Spring 2014. However, the level of 
agreement between two groups at both time points was substantively importantly different. Examination 

of the cell standardized residuals (s.r.) showed that none of the cells individually was a major contributor 
to the overall statistically significant relationship between Phase and response for students in Spring 2012. 
Note that the largest percentage of Phase 1 students responded “No” in both Spring 2012 and Spring 
2014, whereas the largest percentage of Phase 2 students responded “Maybe” in Spring 2012 and “No” 

in Spring 2014. In addition, Phase 2 had a larger percentage of students respond “Yes” at both time 
points. 
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Table 15. Results of Mann-Whitney U Test: Comparison of Houston Middle School Cohort Phase 1 and Phase 2 Students’ Responses 
| Spring 2012 Spring 2014 


Phase 1 Phase 2 Phase 1 Phase 2 
N Mean N Mean Z g N Mean N Mean Z g 
| like science. 146 1.27 97 149 280% -0.37 146 1.04 97 137 4.23" —-0.55 


How often do you talk 
to your family about 
what you do in science 
class? 

How often do you talk 
to your friends about ae rn 

what you do in science 149 0.68 99 0.92 2.87 -0.37 149 0.54 99 0.71 2,13 -0.28 
class? 


*o < 0.05, two-tailed; **p < 0.01, two-tailed; ***p < 0.001, two-tailed. A negative g indicates that Phase 2 has a higher score relative to Phase 1. 


151 0.79 99 0.89 1.26 -0.16 151 0.42 99 0.66 3.33** -0.43 


Table 16. Results of Chi-Square Test: Comparison of Houston Middle School Cohort Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 


Spring 2012 Spring 2014 
(X2(2) = 5.27, p2 = 0.072, Cramer's V = 0.15, g = 0.29) (X2(2) = 6.62, p = 0.036*, Cramer's V = 0.16, g = 0.32) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI. n % Sr n % SI. n % SI. 
Yes 81 54.0 -0.5 61 61.0 0.6 51 34.0 -0.6 42 42.0 0.8 
Maybe 62 41.3 0.2 39 39.0 -0.2 74 49.3 -0.2 52 52.0 0.2 
No 7 47 1.4 0 0.0 -17 25 16.7 15 6 6.0 -18 


Would you like to be a scientist when you are older? 


Spring 2012 Spring 2014 
(X2(2) = 10.69, p = 0.005*, Cramer's V = 0.21, g = 0.41) (X2(2) = 5.36, p = 0.068, Cramer's V = 0.15, g = 0.29) 
Phase 1 Phase 2 Phase 1 Phase 2 
% SI. n % Su. n % Su. n % Su. 
Yes 7 47 -0.9 9 9.2 11 4 2.7 -1.0 7 7.1 13 
Maybe 53 35.6 -1.2 51 52.0 15 31 20.8 -0.8 28 28.6 0.9 
No 89 59.7 14 38 38.8 -17 114 76.5 0.7 63 64.3 -0.9 


Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 


a: Exact test was performed since more than 20% of the cells have expected cell counts less than 5. 
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Houston Results: 
Comparison of Student Attitudes Toward Science 
from Fall 2011 (or Spring 2012) to Spring 2014 


The results of the Wilcoxon Signed-rank test for the Elementary and Middle School cohorts are 
summarized in Table 17 and Table 19, respectively. The results of the marginal homogeneity test for the 
Elementary and Middle School cohorts are summarized in Table 18 and Table 20, respectively. 


Elementary Cohort Results (Figure 3, page 37): Within Group Comparisons 


=» “I like science” 


As shown in Table 17, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher level of agreement) in Fall 2011 than in Spring 
2014 (Z = 3.91, p < 0.001, r = -0.12 and Z = 4.96, p < 0.001, r= -0.17, respectively). However, the 
associated effects were small. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 17, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher frequency) in Fall 2011 than in Spring 2014 (Z = 
5.67, p < 0.001, r=-0.17 and Z = 5.57, p < 0.001, r = -0.19, respectively). However, the associated 
effects were small. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 17, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher frequency) in Fall 2011 than in Spring 2014 (Z = 
4.19, p < 0.001, r= -0.13 and Z = 6.13, p < 0.001, r = -0.2, respectively). However, the associated effects 
were small. 


=" “Do you think science will be useful when you are older?” 


As shown in Table 18, the marginal homogeneity tests showed that there were no statistically significant 
differences between Fall 2011 and Spring 2014 in the level of agreement for this question within both 
Phase 1 and Phase 2 student groups. The level of agreement for Phase 2 students was higher in Spring 
2014, whereas the level of agreement was the higher for Phase 1 students in Fall 2011. 


= “Would you like to be a scientist when you are older?” 


As shown in Table 18, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both Phase 1 and Phase 2 groups from Fall 
2011 to Spring 2014 (MH = 8.84, p < 0.001 and MH = 8.30, p < 0.001, respectively), with the level of 
agreement being higher in Fall 2011 for both groups. 
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Table 17. Comparison of the Distributions of Houston Elementary Cohort Phase 1 and Phase 2 Students’ 


Responses with the Wilcoxon Signed-Rank Test 
Phase 1 Phase 2 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 
N Mean N Mean Z r N Mean N Mean Z r 
| like science. 561 1.68 561 1.55 3.91*** = -0,12 407 1.71 407 154 4.96" -0,17 


How often do you talk 
to your family about ei oe 
what you do in 556 1.13 556 0.91 5.67 -0.17 | 409 1.11 409 0.88 5.57 -0.19 
science class? 

How often do you talk 
to your friends about 
what you do in 
science class? 

*** 9 < 0.001, two-tailed. A negative r indicates that Fall 2011 has a higher score relative to Spring 2014. 


557 0.93 557 0.76 4.19** -0.13 | 405 0.99 405 0.69 8 6.13** = -0.22 


Table 18. Results of Marginal Homogeneity Test: Comparison of the Distributions of Houston Elementary Cohort Students’ 
Responses within the Phase 1 and Phase 2 Groups 


Marginal Homogeneity Test 


N SICUUCOM nm CCL 
Phase 1 568 1.03 0.302 
Do you think science will be useful to you when you are older? 
Phase 2 418 -1.60 0.110 
Phase 1 561 8.84 <0.001* 
Would you like to be a scientist when you are older? 
Phase 2 406 8.30 <0.001* 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative to 
Fall 2011. 
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Middle School Cohort Results (Figure 4, page 39): Within Group Comparisons 
=» “I! like science” 


As shown in Table 19, both Phase 1 and Phase 2 students had slightly higher scores (i.e., a higher level 
of agreement) in Spring 2012 than in Spring 2014. The Wilcoxon Signed-rank test revealed that the 
within-group difference for Phase 1 students was statistically significant (Z = 3.66, p < 0.001, r= -0.21), 
but not for Phase 2 students. However, the associated effect sizes for both Phase 1 and Phase within- 
group differences were small. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 19, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher frequency) in Spring 2012 than in Spring 2014 (Z 
= 5.27, p < 0.001, r = -0.30 and Z = 3.36, p = 0.001, r = -0.24, respectively). The associated effect size 
was medium for the Phase 1 within-group difference and small for the Phase 2 within-group difference. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 19, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher frequency) in Spring 2012 than in Spring 2014 
(Z = 2.09, p = 0.036, r= -0.12 and Z = 2.68, p = 0.007, r= -0.19, respectively). However, the associated 
effect sizes for both the Phase 1 and Phase 2 within-group differences were small. 


* “Do you think science will be useful when you are older?” 


As shown in Table 20, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both Phase 1 and Phase 2 student groups 
between Spring 2012 and Spring 2014 (MH = 4.58, p < 0.001 and MH = 3.57, p < 0.001, respectively), 
with the level of agreement being higher in Spring 2012 for both groups. 


=» “Would you like to be a scientist when you are older?” 


As shown in Table 20, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both Phase 1 and Phase 2 student groups 
between Spring 2012 and Spring 2014 (MH = 3.56, p < 0.001 and MH = 3.64, p < 0.001, respectively), 
with the level of agreement being higher in Spring 2012 for both groups. 


Summative Report Section 5: Student Attitudes 31 


Table 19. Comparison of the Distributions of Houston Middle School Cohort Phase 1 and Phase 2 Students’ 


Responses with the Wilcoxon Signed-Rank Test 
Phase 1 EW 


Spring 2012 Spring 2014 Spring 2012 Spring 2014 
N Mean N Mean Z r N Mean N Mean Z r 
| like science. 146 1.27 146 104 3.66** -0.21 97 1.49 97 1.37 190 -0.14 


How often do you talk 
to your family about 
what you do in science 
class? 

How often do you talk 
to your friends about 
what you do in science 
class? 

*o < 0.05, two-tailed; ** p < 0.01, two-tailed; *** p < 0.001, two-tailed. A negative r indicates that Fall 2011 has a higher score relative to 
Spring 2014. 


151 0.79 151 042 5.27*** = -0.30 99 0.89 99 0.66 3.36 -0.24 


149 0.68 149 0.54 2.09* -0.12 99 0.92 99 0.71 2.68 -0.19 


Table 20. Results of Marginal Homogeneity Test: Comparison of the Distributions of Houston Middle School Cohort Students’ 
Responses within the Phase 1 and Phase 2 Groups 


Marginal Homogeneity Test 


N SI CUUC WOM aS CLC 
Phase 1 150 4.58 <0,001* 
Do you think science will be useful to you when you are older? 
Phase 2 100 3.57 <0.001* 
Phase 1 149 3.56 <0.001* 
Would you like to be a scientist when you are older? 
Phase 2 98 3.64 <0.001* 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative 
to Fall 2011. 
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Figure 3. Student Attitude Survey, Houston: Fall 2011 and Spring 2014: Elementary Cohort Question Responses by Group 


| like Science ... 
Not at all A little A lot 
Phase 1 EE" 23.4% 
Fall 2011 


Phase 2 ER) 224% 


Not at all A little A lot 
Phase 1 EE 35.7% 
Spring 2014 


Phase 2 39.6% 


How often do you talk to your family about what you do in science class? 


Never or almost never Some A lot 
Phase 45.9% 
Fall 2011 


Phase 2 47.9% 


Never or almost never Some Alot 
Phase 1 61.7% 
Spring 2014 


Phase 2 64.3% 
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How often do you talk to your friends about what you do in science class? 
Never or almost never Some Alot 

Phase 1 32.7% 41.8% 25.5% 

Fall 2011 


ae 43.0% 27.9% 


Never or almost never Some Alot 


Phase 1 38.2% 47.9% 13.8% 
Spring 2014 


Phase 52.8% 2.2% 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase 1 ER, 43.5% 
Fall 2011 
Phase2 EE} 37.8% 
No Maybe Yes 
Phase 1 ER 40.0% 
Spring 2014 
Phase 2 3 48.0% 


Would you like to be a scientist when you are older? 
No Maybe Yes 


Phase 1 33.5% 33.7% 32.8% 
Fall 2011 


Phase 2 36.7% 32.3% 


No Maybe Yes 
Phase 1 39.9% 
Spring 2014 
Phase 2 51.5% 38.2% 
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Figure 4. Student Attitude Survey, Houston: Fall 2011 and Spring 2014: Middle School Cohort Question Responses by Group 


| like Science ... 


Not at all 


Phase 1 


Phase 2 i 


Not at all 


Phase 1 


Phase 2 B 


A little A lot 
57.5% 
Spring 2012 


42.5% 


A little Alot 


65.8% 19.2% 


Spring 2014 


56.7% 


How often do you talk to your family about what you do in science class? 


Never or almost never Some A lot 


Phase 1 36.4% 47.7% 


Spring 2012 
Phase 2 24.2% 62.6% 
Never or almost never Some Alot 


Phase 1 36.4% 


Spring 2014 


Phase 2 55.6% 
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How often do you talk to your friends about what you do in science class? 


Never or almost never Some Alot 
Phase 1 47.7% 
Spring 2012 


Phase 2 25.3% 57.6% 


Never or almost never Some A lot 

Phase 1 53.0% 40.3% 6.7% 
Spring 2014 

Phase 2 39.4% 50.5% 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase EE 41.3% 
Spring 2012 


Phase 2 39.0% 


No Maybe Yes 


Phase 49.3% 


Spring 2014 


Phase 2 Et 52.0% 


Would you like to be a scientist when you are older? 


No Maybe Yes 

Phase 1 59.7% 35.6% 
Spring 2012 

Phase 2 38.8% 52.0% 

No Maybe Yes 

Phase 1 20.8% 
Spring 2014 

Phase 2 64.3% 28.6% 
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Overall Summary of Houston PASS Student Attitudes Results 


Between-qroup difference 


For the Elementary Cohort in HISD, on the three Likert-scale items (‘I like science”, “How often do you 
talk to your family about what you do in science class?”, and “How often do you talk to your friends about 
what you do in science class?”), none of the items examined exhibited either statistically significant or 
substantively meaningful differences between Phase 1 and Phase 2 students either at the baseline (Fall 
2011) or in Spring 2014. However, for the Middle School Cohort, statistically significant and substantively 
meaningful differences between Phase 1 and Phase 2 students were found on two of the items in both 
Spring 2012 and Spring 2014: “I like science” and “How often do you talk to your friends about what you 
do in science class?”, both of which favored Phase 2 at both time points. Meanwhile, while there was no 
statistically or substantively meaningful difference between the groups in Spring 2012 on the item “How 
often do you talk to your family about what you do in science class?”, there was both a statistically 
significant and substantively meaningful difference in Spring 2014 that favored Phase 2 (g = -0.43). 


For the two nominal scale items (“Do you think science will be useful to you when you are older?” and 
‘Would you like to be a scientist when you are older?”), both groups in the Elementary Cohort had similar 
attitudes at both time points: neither statistically significant nor substantively meaningful differences 
between Phase 1 and Phase 2 students were found except for the difference on the first question in 
Spring 2014. For this question, there was a statistically significant difference between the two groups, but 
the difference was not substantively meaningful, with Phase 1 having a higher percentage of students 
responding “Yes” (54.4% vs. 48.8%). Unlike the Elementary Cohort, there were substantively meaningful 
differences between Phase land Phase 2 students in the Middle School Cohort for these two questions 
at both time points, although the differences were statistically significant only for the first question in 
Spring 2014 and the second question in Spring 2012. 


In summary, where there was baseline equivalence between Phase 1 and Phase 2 students, there were 
only two statistically significant outcomes. For the first, Phase 1 Elementary Cohort students had a higher 
percentage of students respond “Yes” (54.4% vs. 48.8%) to the question “Do you think science will be 
useful to you when you are older?”, but the difference was not substantively important. For the second, 
Phase 2 Middle School Cohort students scored statistically significantly higher than Phase 1 students on 
the question “How often do you talk to your family about what you do in science class?”, with an effect 
size (g = 0.43) that was substantively important. 


Within-group difference 


For the three Likert-scale items, both Phase 1 and Phase 2 students in the Elementary Cohort liked 
science more or were more likely to talk to their friends or families about science in Fall 2011 compared to 
Spring 2014, with the changes within both groups being statistically significant. However, all of the 
changes were considered to be small. For the Middle School Cohort, both groups liked science more or 
were more likely to talk to their friends or families about science in Spring 2012 compared to Spring 2014. 
All differences were statistically significant except for the responses of Phase 2 students to the item “I like 
science”. However, the only meaningful change was found for Phase 1 students on the item “How often 
do you talk to your family about what you do in science class?”, which had a medium effect size. 


For the nominal scale item “Do you think science will be useful to you when you are older?”, only Phase 1 
and Phase 2 students in the Middle School Cohort demonstrated statistically significant differences from 
Spring 2012 to Spring 2014, with a higher percentage of agreement in Spring 2012 for both groups. For 
the other nominal scale item “Would you like to be a scientist when you are older?”, both groups had a 
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statistically significantly higher percentage of agreement at the baseline (Fall 2011 or Spring 2012). 
However, since no effect size measure for the marginal homogeneity test was available, we do not know 
whether these statistically significant differences were substantively important. 
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New Mexico Region: 
Results for Spring 2014 PASS 
Student Attitudes Toward Science 
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New Mexico 
Spring 2014 PASS Student Attitudes 
Key Findings for Phase 1 


For all students combined in the New Mexico region, the following outcomes favoring Phase 1 students 
were found on the Spring 2014 PASS Student Attitudes section. 


For the between-group differences, on the two nominal questions (“Do you think science will be 
useful to you when you are older?” and “Would you like to be a scientist when you are older?”), 
the only meaningful difference between Phase 1 and Phase 2 students was found for the Middle 
School Cohort in Spring 2014 on the item “Would you like to be a scientist when you are older?”, 
where Phase 1 students were more likely to respond “Yes” or “Maybe” (29% total) compared to 
Phase 2 (15.2% total). In addition, while the difference did not reach a substantively meaningful 
level, there was a statistically significant difference for the Elementary Cohort in Spring 2014 on 
the item “Do you think science will be useful when you are older?”, with a higher percentage of 
Phase 1 students responding “Yes” (57.2% vs. 47.5%). Furthermore, while the largest 
percentage of Phase 1 students responded “Yes” at both time points to this question, the largest 
percentage of Phase 2 students responded “Yes” in Fall 2011 and “Maybe” in Spring 2014. 
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New Mexico: 
Fall 2011 to Spring 2014 PASS Student Attitudes Results 


PASS student attitudes survey data results in the New Mexico region from the Fall 2011 (baseline or pre- 
intervention) and Spring 2014 (third posttest) administrations are currently available and are reported 
below. 


Survey Questions: Student Attitudes Toward Science 


Of the 14 total attitude questions on the student survey, the five questions related to student attitudes 
towards science have been selected for discussion. See Table A - 5and Table A - 6 in Appendix A for the 
outcomes on all 14 student attitudes questions. Table 21 and Table 22 show the final analytic sample 
sizes included for the New Mexico region for the Elementary Cohort and Middle School Cohort 
respectively. 


Table 21. PASS-Basic, New Mexico, Spring 2014: Samples for the Survey Analyses for Elementary Cohort 
Sample Phase 1 Phase 2 


Initial Samples? 511 426 
| like science. 474 280 
How often do you talk to your family about what you do in science class? 462 280 
How often do you talk to your friends about what you do in science class? 452 277 
Do you think science will be useful when you are older? 481 284 
Would you like to be a scientist when you are older? 470 278 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 


Table 22. PASS-Basic, New Mexico, Spring 2014: Samples for the Survey Analyses for Middle School Cohort 
Sample Phase 1 Phase 2 


Initial Samples? 502 145 
| like science. 373 84 
How often do you talk to your family about what you do in science class? 376 79 
How often do you talk to your friends about what you do in science class? 377 7 
Do you think science will be useful when you are older? 376 82 
Would you like to be a scientist when you are older? 372 79 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 
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Fall 2011 & Spring 2014 New Mexico Results: 
Comparison of Phase 1 and Phase 2 Student Attitudes Toward Science 


The results of the Mann-Whitney U test for the Elementary Cohort and the Middle School Cohort are 
summarized in Table 23 and Table 25, respectively. The results of the chi-square test of independence 
for the Elementary Cohort and the Middle School Cohort are summarized in Table 24 and Table 26 
respectively. 


Elementary Cohort Results (Figure 5, page 57): Between Group Comparisons) 


» “I like science” 


As shown in Table 23, the Mann-Whitney U test results revealed that Phase 1 students had statistically 
significantly and substantively importantly higher scores (i.e., a higher level of agreement) than Phase 2 
students in Fall 2011 (Z = -3.59, p < 0.001, g = 0.27). In contrast, although Phase 1 students had higher 
scores than Phase 2 students in Spring 2014, the difference was neither statistically significant nor 
substantively important. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 23, the Mann-Whitney U test results revealed that Phase 1 students had statistically 
significantly and substantively higher scores (i.e., a higher level of agreement) than Phase 2 students in 
Fall 2011 (Z = -3.32, p = 0.001, g = 0.25). Meanwhile although Phase 1 students had higher scores than 
Phase 2 students in Spring 2014, the difference was neither statistically significant nor substantively 
important. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 23, the Mann-Whitney U test results revealed that Phase 1 students had higher scores 
(i.e., a higher frequency) than Phase 2 students in both Fall 2011 and Spring 2014, with the difference in 
Spring 2014 being statistically significant (Z = -2.34, p = 0.019, g = 0.18). However, neither of the 
associated effect sizes was substantively important. 


=" “Do you think science will be useful when you are older?” 


As shown in the top section of Table 24, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was statistically significantly different in both Fall 
2011 and Spring 2014 (77(2) = 10.32, p = 0.006, g = 0.23 and x7(2) = 7.16, p = 0.028, g = 0.19, 
respectively). However, neither difference was substantively important. Examination of the cell 
standardized residuals (s.r.) showed that none of the cells individually was a major contributor to the 
overall statistically significant relationship between Phase and response for students at either time point. 
Rather, they worked jointly to contribute to the overall statistical significance, with a higher percentage of 
Phase 1 students responding “Yes” in both Fall 2011 (61.3% vs. 50.7%) and Spring 2014 (57.2% vs. 
47.5%). In addition, while the largest percentage of Phase 1 students responded “Yes” at both time 
points, the largest percentage of Phase 2 students responded “Yes” in Fall 2011 and “Maybe” in Spring 
2014. 


=» “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 24, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was not statistically significantly 
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different in Spring 2014, whereas the difference was statistically significant in Fall 2011 (x? (2) = 8.98, p = 
0.011, g = 0.22). However, neither difference was substantively important. Examination of the cell 
standardized residuals (s.r.) showed that none of the cells individually was a major contributor to the 
overall statistically significant relationship between Phase and response for students in Fall 2011. Rather, 
they worked jointly to contribute to the overall statistical significance, with a higher percentage of Phase 1 
students responding “Yes” (24.9% vs. 19.1%). Note that in Fall 2011, the largest percentage of Phase 1 
students responded “Maybe”, while the largest percentage of Phase 2 students responded “No”. In Spring 
2014, the largest percentage of both Phase 1 and Phase 2 students responded “No” (50.2% vs. 53.2%). 


Table 23. Results of Mann-Whitney U Test: Comparison of New Mexico Elementary Cohort Phase 1 and Phase 2 Students’ Responses 
Fall 2011 Spring 2014 


Phase 1 Phase 2 Phase 1 Phase 2 
N Mean N Mean Z g N Mean N Mean Z g 
| like science. 474, 1.74 280 162 -3.59"* 0.27 | 474 1.54 280 151 -0.89 0.07 


How often do you talk to 
your family about what 462 1.07 280 0.89 3.32 0.25 | 462 0.90 280 0.87 -0.70 0.05 
you do in science class? 
How often do you talk to 
your friends about what 452 ~—-0.70 277 0.64 -0.96 0.07 452 0.70 277 0.59 
you do in science class? 
** 9 < 0.05, two-tailed; *** p < 0.001, two-tailed. A negative g indicates that Phase 2 has a higher score relative to Phase 1. 


0.18 


Table 24, Results of Chi-Square Test: Comparison of New Mexico Elementary Cohort Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 


Fall 2011 Spring 2014 
(X2(2) = 10.32, p = 0.006*, Cramer's V = 0.12, g = 0.23) (X2(2) = 7.16, p = 0.028*, Cramer's V = 0.10, g = 0.19) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % Sw. n % Sw. n % Sw. n % Sw. 
Yes 295 61.3 11 144 50.7 “15 275 57.2 11 135 47.5 -1.4 
Maybe 172 35.8 -1.0 123 43.3 1.3 185 =. 38.5 -1.2 137 48.2 1.6 
No 14 2.9 -1.2 17 6.0 1.6 21 44 0.1 12 42 -0.1 
Would you like to be a scientist when you are older? 
Fall 2011 Spring 2014 
(X2(2) = 8.98, p = 0.011*, Cramer's V = 0.11, g = 0.22) (X2(2) = 1.44, p = 0.487, Cramer's V = 0.04, g =0.09) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % Si n % SI n % Su n % SI 
Yes 117 24.9 1.0 53 19.1 -1.3 66 14.0 0.6 31 11.2 -0.8 
Maybe 178 37.9 0.7 91 32.7 -0.9 168 =. 35.7 0.0 99 35.6 0.0 
No 175 37.2 -14 134 = 48.2 18 236 = 50.2 -0.3 148 53.2 0.4 


Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 
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Middle School Cohort Results (Figure 6, page 59): Between Group Comparisons) 


» “I like science” 


As shown in Table 25, the Mann-Whitney U test results revealed that Phase 1 students had both 
statistically significantly and substantively higher scores (i.e., a higher level of agreement) than Phase 2 
students in both Fall 2011 and Spring 2014 (Z = -2.27, p = 0.023, g = 0.27 and Z = -2.04, p = 0.042, g= 
0.25, respectively). 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 25, Phase 1 students had slightly higher scores (i.e., a higher frequency) than Phase 2 
students in Spring 2014, whereas Phase 1 and Phase 2 students had almost the same scores in Fall 
2011. However, the Mann-Whitney U test results revealed such differences were neither statistically 
significantly different nor substantively important in either Fall 2011 or Spring 2014. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 25, Phase 1 students had higher scores (i.e., a higher frequency) than Phase 2 
students in Spring 2014, whereas Phase 2 students had slightly higher scores in Fall 2011. However, the 
Mann-Whitney U test results revealed such differences were neither statistically significantly different nor 
substantively important in either Fall 2011 or Spring 2014. 


* “Do you think science will be useful when you are older?” 


As shown in the top section of Table 26, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was neither statistically significant nor 
substantively important in either Fall 2011 or Spring 2014. The largest percentage of Phase 1 and Phase 
2 students in Fall 2011 responded “Yes”, whereas the largest percentage of Phase 1 and Phase 2 
students in Spring 2014 responded “Maybe”. 


= “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 26, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was neither statistically significant 
nor substantively important in Fall 2011; however, the level of agreement was both statistically significant 
and substantively important in Spring 2014 (y7(2) = 8.06, p = 0.018, g = 0.27). Examination of the cell 
standardized residuals (s.r.) showed that “Maybe” response by Phase 2 students was a major contributor 
to the overall significant relationship between Phase and students’ response in Spring 2014, with Phase 2 
having fewer than expected “Maybe” responses. The largest percentage of Phase 1 and Phase 2 
students at both time points responded “No”, with the percentage choosing “No” being higher in Spring 
2014. 
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Table 25. Results of Mann-Whitney U Test: Comparison of New Mexico Middle School Cohort 


Phase 1 and Phase 2 Students’ Responses 
Fall 2011 Spring 2014 


Phase 1 Phase 2 Phase 1 Phase 2 
N Mean N Mean Z g N Mean N Mean Z g 
| like science. 373 1.54 84 139 -2.27* =0.27 373 1.16 84 101 = -2.04* 0.25 


How often do you talk to 
your family about what you 376 0.99 79 0.99 -0.09 0.01 376 0.57 19 0.44 -1.49 0.18 
do in science class? 

How often do you talk to 
your friends about what you 377 0.72 77 0.74 0.30 -0.04 377 0.59 7 0.48 -1.41 0.18 
do in science class? 
*o < 0.05, two-tailed. A negative g indicates that Phase 2 has a higher score relative to Phase 1. 


Table 26. Results of Chi-Square Test: Comparison of New Mexico Middle School Cohort 
Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 


Fall 2011 Spring 2014 
(X2(2) = 0.43, p = 0.809, Cramer's V = 0.03, g = 0.06) (X2(2) = 4.40, p = 0.111, Cramer's V = 0.10, g = 0.20) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI. n % Sf. n % SI. n % Sf. 
Yes 212 56.4 0.2 43 52.4 -0.4 125 33.2 0.5 21 25.6 -1.0 
Maybe 147 39.1 -0.2 35 42.7 0.4 186 = 49.5 -0.6 51 62.2 13 
No 17 45 -0.1 4 49 0.1 65 17.3 0.4 10 12.2 -0.9 
Would you like to be a scientist when you are older? 
Fall 2011 Spring 2014 
(X2(2) = 2.08, p = 0.354, Cramer's V = 0.07, g = 0.14) (X2(2) = 8.06, p = 0.018*, Cramer's V = 0.13, g = 0.27) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI. n % SI. n % Sf. n % SI. 
Yes 34 9.1 0.5 4 5.1 -1.0 16 43 -0.1 4 5.1 0.3 
Maybe 148 39.8 0.2 29 36.7 -0.4 92 24.7 1.0 8 10.1 -2.3 
No 190 51.1 -0.3 46 58.2 0.7 264 = 71.0 -0.5 67 84.8 1.2 


Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 
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New Mexico Results: 
Comparison of Student Attitude Toward Science 
from Fall 2011 to Spring 2014 


The results of the Wilcoxon Signed-rank test for the Elementary Cohort and Middle School Cohort are 
summarized in Table 27 and Table 29, respectively. The results of the marginal homogeneity test for the 
Elementary Cohort and Middle School Cohort are summarized in Table 28 and Table 30, respectively. 


Elementary Cohort Results (Figure 5, page 57): Within Group Comparisons 


» “I like science” 


As shown in Table 27, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 6.14, p < 0.001, r = -0.20 
and Z = 2.67, p = 0.008, r= -0.11, respectively). Furthermore, the associated effect size for both Phase 1 
and Phase 2 within-group difference were small in magnitude. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 27, both Phase 1 and Phase 2 students had higher scores (i.e., a higher frequency) in 
Fall 2011 than in Spring 2014. The Wilcoxon Signed-rank test revealed that the within-group difference 
was not statistically significant for Phase 2 students, but was statistically significant for Phase 1 students 
(Z = 4.13, p < 0.001, r= -0.14). Furthermore, the associated effect size for the Phase 1 within-group 
difference was small in magnitude, while the difference for Phase 2 was Negligible. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 27, Phase 2 students had slightly higher scores (i.e., a higher frequency) in Fall 2011 
than in Spring 2014. Phase 1 students had almost the same scores at both time points. The Wilcoxon 
Signed-rank test suggested that the within-group differences for both Phase 1 and Phase 2 students were 
not statistically significant. Furthermore, the associated effect sizes for both the Phase 1 and Phase 2 
within-group differences were Negligible in magnitude. 


=" “Do you think science will be useful when you are older?” 


As shown in Table 28, the marginal homogeneity tests showed that there were no statistically significant 
differences in the level of agreement for this question within either the Phase 1 or Phase 2 student groups 
from Fall 2011 to Spring 2014. The level of agreement was higher in Fall 2011 for both groups. 


=» “Would you like to be a scientist when you are older?” 


As shown in Table 28, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both the Phase 1 and Phase 2 groups from 
Fall 2011 to Spring 2014 (MH = 5.61, p < 0.001 and MH = 2.55, p = 0.011, respectively), with the level of 
agreement being higher in Fall 2011 for both groups. 


Summative Report Section 5: Student Attitudes 46 


Table 27. Results of Wilcoxon Signed-Rank Test: Comparison of New Mexico Elementary Cohort Students’ 


Responses within the Phase 1 and Phase 2 Groups 
Phase 1 Phase 2 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 
N Mean N Mean Z r N Mean N Mean Z r 

| like science. 474 174 474 154 614** = -0.20 280 162 280 151 267" -0.11 
How often do you talk to 

your family about what you 462 107 4462 090 413** -0.14 280 0.89 280 0.87 0.47 ~—-0.02 
do in science class? 

How often do you talk to 

your friends about what 452 0.70 452 0.70 -0.01 0.00 277 0.64 277 ~# 0,59 103 = -0.04 


you do in science class? 


* 9 < 0.01; ** p < 0.001, two-tailed. A negative r indicates that Fall 2011 has a higher score relative to Spring 2014. 


Table 28. Results of Marginal Homogeneity Test: Comparison of the Distributions of New Mexico Elementary Cohort Students’ 


Responses within the Phase 1 and Phase 2 Groups 


Marginal Homogeneity Test 


N SICUUCUOM SI CLC 
Phase 1 481 1.62 0.105 
Do you think science will be useful to you when you are older? 
Phase 2 284 0.30 0.762 
Phase 1 470 5.61 <0.001* 
Would you like to be a scientist when you are older? 
Phase 2 278 2.55 0.011* 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative to 


Fall 2011. 
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Middle School Cohort Results (Figure 6, page 59): Within Group Comparisons 


» “I like science” 


As shown in Table 29, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 8.46, p < 0.001, r = -0.31 
and Z = 4.26, p < 0.001, r = -0.33, respectively). The associated effect size for both the Phase 1 and 
Phase 2 within-group difference was medium in magnitude. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 29, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher frequency) in Fall 2011 than in Spring 2014 (Z = 
8.95, p < 0.001, r = -0.33 and Z = 5.48, p < 0.001, r = -0.44, respectively). The associated effect size for 
both the Phase 1 and Phase 2 within-group difference was medium in magnitude. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 29, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher frequency) in Fall 2011 than in Spring 2014 (Z = 
3.11, p = 0.002, r= -0.11 and Z = 2.40, p = 0.016, r = -0.19, respectively). The associated effect size for 
both the Phase 1 and Phase 2 within-group difference was small in magnitude. 


=" “Do you think science will be useful when you are older?” 


As shown in Table 30, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both the Phase 1 and Phase 2 groups from 
Fall 2011 to Spring 2014 (MH = 7.76, p < 0.001 and MH = 3.88, p < 0.001, respectively), with the level of 
agreement being higher in Fall 2011 for both groups. 


=» “Would you like to be a scientist when you are older?” 


As shown in Table 30, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both the Phase 1 and Phase 2 groups from 
Fall 2011 to Spring 2014 (MH = 6.32, p < 0.001 and MH = 3.20, p = 0.001, respectively), with the level of 
agreement being higher in Fall 2011 for both groups. 
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Table 29. Results of Wilcoxon Signed-Rank Test: Comparison of New Mexico Middle School Cohort Students’ 


Responses within the Phase 1 and Phase 2 Groups 
Phase 1 Phase 2 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 
N Mean N Mean Z r N Mean N Mean Z r 

| like science. 373 1540 0=— 3731.16) 8. 46** = -0.31 84 1.39 84 101 4.26" = -0.33 
How often do you talk to 

your family about what you 376 0.99 376 O57 8.95** -0.33 79 0.99 79 0.44 548** = -0.44 
do in science class? 

How often do you talk to 

your friends about what 377 0.72377 0598-011 77 0.74 77 0.48 2,40* -0.19 


you do in science class? 


* p < 0.05, two-tailed; ** p < 0.01; *** p < 0.001, two-tailed. A negative r indicates that Fall 2011 has a higher score relative to Spring 2014. 


Table 30. Results of Marginal Homogeneity Test: Comparison of the Distributions of New Mexico Middle School Cohort Students’ 


Responses within the Phase 1 and Phase 2 Groups 


Marginal Homogeneity Test 


N SICUUCUOM SI ClCL Ce 
Phase 1 376 7.76 <0.001* 
Do you think science will be useful to you when you are older? 
Phase 2 82 3.88 <0.001* 
Phase 1 372 6.32 <0.001* 
Would you like to be a scientist when you are older? 
Phase 2 79 3.20 0.001* 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative to 


Fall 2011. 
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Figure 5. Student Attitude Survey, New Mexico: Fall 2011 and Spring 2014: Elementary Cohort Question Responses by Group 


| like Science ... 


Notatall  Alittle A lot 


Phase1 20.9% 76.8% 
Fall 2011 


Phase 2 Bc sils3iex 


Not at all A little A lot 


Phase 1 38.0% 


Spring 2014 


Phase 2 41.8% 


How often do you talk to your family about what you do in science class? 


Never or almost never Some A lot 
Phase 1 54.8% 
Fall 2011 


Never or almost never Some A lot 
Phase 1 60.0% 15.2% 


Spring 2014 


Phase 2 62.5% 12.1% 
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How often do you talk to your friends about what you do in science class? 


Never or almost never Some Alot 
Phase 1 47.6% 35.0% 
Fall 2011 


Phase 2 50.5% 34.7% 


Never or almost never Some Alot 
Phase 1 EZ 48.7% 
Spring 2014 


Phase 43.7% 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase 1 35.8% 
Fall 2011 


Phase 2 Ey) 43.3% 


No Maybe Yes 
Phase 1 EZ 38.5% 
Spring 2014 


Phase 2 [Ey 48.2% 


Would you like to be a scientist when you are older? 


No Maybe Yes 
Phase 1 37.2% 37.9% 
Fall 2011 


Phase 32.7% 


No Maybe Yes 
Phase 1 35.7% 
Spring 2014 
Phase 2 53.2% 35.6% 
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Figure 6. Student Attitude Survey, New Mexico: Fall 2011 and Spring 2014: Middle School Cohort Question Responses by Group 


| like Science ... 
Not at all A little A lot 
Phase 1 E 42.6% 
Fall 2011 


Phase 2 56.0% 


Not at all A little Alot 
Phase 1 55.8% 
Spring 2014 


Phase 70.2% 


How often do you talk to your family about what you do in science class? 


Never or almost never Some Alot 
Phase 1 56.9% 
Fall 2011 


Phase 2 (NNPTEE 58.2% 


Never or almost never Some A lot 
Phase 4 42.3% 
Spring 2014 
Phase 2 58.2% 39.2% P| 
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How often do you talk to your friends about what you do in science class? 


Never or almost never Some Alot 

Phase 1 46.7% 
Fall 2011 

Phase 2 39.0% 48.1% 

Never or almost never Some Alot 

Phase 1 47.8% 
Spring 2014 

Phase 2 54.5% 42.9% | 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase EX 39.1% 
Fall 2011 


Phase 2 ER 42.7% 


No Maybe Yes 
Phase 4 49.5% 
Spring 2014 


Phase 2 PET 62.2% 


Would you like to be a scientist when you are older? 


No Maybe Yes 
Phase 1 39.8% 
Fall 2011 
Phase 2 58.2% 36.7% 
No Maybe Yes 
Phase 1 24.7% 
Spring 2014 
Phase 2 84.8% 0.1% 
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Overall Summary of New Mexico PASS Student Attitudes Results 


Between-qroup difference 


For two of the three Likert-scale items (“I like science” and “How often do you talk to your family about 
what you do in science class?”, Phase 1 students in both the Elementary and Middle School cohorts in 
the New Mexico region had higher levels of agreement at both time points compared to Phase 2 students. 
On the third Likert-scale item (“How often do you talk to your friends about what you do in science 
class?”), while Phase 1 students in the Elementary Cohort also had higher levels of agreement in both 
Fall 2011 and Spring 2014, Phase 2 students in the Middle School Cohort were more likely to talk to their 
friends in Fall 2011 relative to Phase 1 students, but not in Spring 2014. Furthermore, for the Elementary 
Cohort, the first two items exhibited meaningful differences between Phase 1 and Phase 2 students at the 
baseline (Fall 2011), but not in Spring 2014. However, for the Middle School Cohort, the item “I like 
science” revealed meaningful differences favoring Phase 1 students at both the baseline (Fall 2011) and 
in Spring 2014. 


For the two nominal questions (“Do you think science will be useful to you when you are older?” and 
“Would you like to be a scientist when you are older?”), the only meaningful difference between Phase 1 
and Phase 2 students was found for the Middle School Cohort in Spring 2014 on the item “Would you like 
to be a scientist when you are older?”, where Phase 1 students were more likely to respond “Yes” or 
“Maybe” (29% total) compared to Phase 2 (15.2% total). While the difference did not reach a 
substantively meaningful level, there was a statistically significant difference for the Elementary Cohort in 
Spring 2014 on the item “Do you think science will be useful when you are older?”, with a higher 
percentage of Phase 1 students responding “Yes” (57.2% vs. 47.5%). In addition, while the largest 
percentage of Phase 1 students responded “Yes” at both time points to this question, the largest 
percentage of Phase 2 students responded “Yes” in Fall 2011 and “Maybe” in Spring 2014. 


Within-group difference 


For the three Likert-scale items (‘I like science”, “How often do you talk to your family about what you do 
in science class?”, and “How often do you talk to your friends about what you do in science class?”), 
Phase 1 and Phase 2 students in both the Elementary and Middle School cohorts had higher levels of 
agreement in Fall 2011 compared to Spring 2014, except for the Elementary Cohort on the item “How 
often do you talk to your friends about what you do in science class?”, where the levels were essentially 
the same at both time points. For the Elementary Cohort, the attitudinal change within both groups from 
Fall 2011 to Spring 2014 was statistically significant only for the item “I like science”, with the level of 
agreement towards this item dropping for both groups. However, the magnitudes of these changes were 
small. In addition, the attitude change from Fall 2011 to Spring 2014 on the item “How often do you talk to 
your family about what you do in science class?” within Phase 1 group was statistically significant, being 
higher in Fall 2011, but was small and not substantively meaningful. The attitudinal changes from Fall 
2011 to Spring 2014 within both groups in the Middle School Cohort were statistically significant on all 
three questions. However, the magnitudes of these changes within both groups were only meaningful, 
with Medium effect sizes, for questions “I like science” and “How often do you talk to your family about 
what you do in science class?” 


For the two nominal scale items (“Do you think science will be useful to you when you are older?” and 
“Would you like to be a scientist when you are older?”), both cohorts on both questions had higher levels 
of agreement in Fall 2011 vs. Spring 2014. The differences were all statistically significant except for both 
Phase 1 and Phase 2 students in the Elementary Cohort on the question “Do you think science will be 
useful to you when you are older?”. However, since no effect size measure for the marginal homogeneity 
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test was available, we do not know whether these statistically significant differences were substantively 
important. 
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North Carolina Region: 
Results for Spring 2014 PASS 
Student Attitudes Toward Science 
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North Carolina 
Spring 2014 PASS Student Attitudes 
Key Findings for Phase 1 


For all students combined in the North Carolina region, the following outcomes favoring Phase 1 students 
were found on the Spring 2014 PASS Student Attitudes section. 


e For the between-group differences, where there was baseline equivalence between Phase 1 and 
Phase 2 students, there was only one statistically significant difference between groups in Spring 
2014 for the Elementary Cohort on the question “How often do you talk to your friends about what 
you do in science class?”, but the difference was not substantively important. 


e For the within-group differences, while not statistically significant, the Phase1 Elementary Cohort 
had higher level of agreement in Spring 2014 vs. the baseline (Fall 2011) on the item “Do you 
think science will be useful to you when you are older?” 
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North Carolina: 
Fall 2011 to Spring 2014 PASS Student Attitudes Results 


PASS student attitudes survey data results in the North Carolina region from the Fall 2011 (baseline or 
pre-intervention) and Spring 2014 (third posttest) administrations are currently available and are reported 
below. 


Survey Questions: Student Attitudes Toward Science 


Of the 14 total attitude questions on the student survey, the five questions related to student attitudes 
towards science have been selected for discussion. See Table A - 7and Table A - 8 in Appendix A for the 
outcomes on all 14 student attitudes questions. Table 31 and Table 32 show the final analytic sample 
sizes included in the North Carolina region for the Elementary Cohort and Middle School Cohort 
respectively. 


Table 31. PASS-Basic, North Carolina, Spring 2014: Samples for the Survey Analyses for the Elementary Cohort 


Sample Phase 1 Phase 2 
Initial Samples? 1,138 975 
| like science. 1,029 871 
How often do you talk to your family about what you do in science class? 1,049 870 
How often do you talk to your friends about what you do in science class? 1,029 870 
Do you think science will be useful when you are older? 1,050 883 
Would you like to be a scientist when you are older? 1,037 867 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 


Table 32. PASS-Basic, North Carolina, Spring 2014: Samples for the Survey Analyses for the Middle School Cohort 


Sample Phase 1 Phase 2 
Initial Samples? 552 1,031 
| like science. 510 899 
How often do you talk to your family about what you do in science class? 513 928 
How often do you talk to your friends about what you do in science class? 510 923 
Do you think science will be useful when you are older? 518 929 
Would you like to be a scientist when you are older? 508 924 


1 The number of students who answered at least one Student Attitudes question in Spring 2014 
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Fall 2011 & Spring 2014 North Carolina Results: 
Comparison of Phase 1 and Phase 2 Student Attitudes Toward Science 


The results of the Mann-Whitney U test for the Elementary Cohort and Middle School Cohort are 
summarized in Table 33 and Table 35, respectively. The results of the chi-square test of independence 
for the Elementary Cohort and Middle School Cohort are summarized in Table 34 and Table 36, 
respectively. 


Elementary Cohort Results (Figure 7, page 77): Between Group Comparisons 
=» “I like science” 


As shown in Table 33, Phase 1 students had higher scores (i.e., a higher level of agreement) than Phase 
2 students in Spring 2014, whereas Phase 2 students had higher scores in Fall 2011. However, the 
Mann-Whitney U test results revealed such differences were neither statistically significantly different nor 
substantively important in either Fall 2011 or Spring 2014. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 33, Phase 1 students had slightly higher scores (i.e., a higher frequency) than Phase 2 
students in both Fall 2011 and Spring 2014. However, Mann-Whitney U test results revealed such 
differences were neither statistically significantly different nor substantively important at either time point. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 33, the Mann-Whitney U test results revealed that Phase 1 students had statistically 
significantly higher scores (i.e., a higher frequency) than Phase 2 students in both Fall 2011 and Spring 
2014(Z = -2.31, p= 0.021, g = 0.11 and Z = -2.96, p = 0.003, g = 0.14, respectively). However, neither of 
the associated effect sizes was substantively important. 


=" “Do you think science will be useful when you are older?” 


As shown in the top section of Table 34, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was neither statistically significantly different nor 
substantively important in either Fall 20110r Spring 2014. Note that the largest percentage of Phase 1 
students responded “Yes” in both Fall 2011 and Spring 2014, whereas the largest percentage of Phase 2 
students responded “Maybe” in Fall 2011 and “Yes” in Spring 2014. 


=» “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 34, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was not statistically significantly 
different in Spring 2014, whereas the difference was statistically significant in Fall 2011 (7(2) = 10.52, p = 
0.005, g = 0.15). However, neither difference was substantively important. Examination of the cell 
standardized residuals (s.r.) showed that none of the cells individually was a major contributor to the 
overall statistically significant relationship between Phase and response for students in Fall 2011. Rather, 
they worked jointly to contribute to the overall statistical significance, with a higher percentage of Phase 1 
students responding “Yes” (21.1% vs. 16.0%). Note that more than half of Phase 1 and Phase 2 students 
responded “No” at both time points. 
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Table 33. Results of Mann-Whitney U Test: Comparison of North Carolina Elementary Cohort 


Phase 1 and Phase 2 Students’ Responses 
Fall 2011 Spring 2014 


Phase 1 Phase 2 


Phase 1 Phase 2 

N Mean N Mean Z g N Mean N Mean Z g 
| like science. 1029 153 871 1.55 0.85 -0.04 ; 1029 151 871 1.48 -1.26 0.06 
How often do you talk to 
your family about what you 1,049 099 870 097 -037 0.02 | 1,049 0.92 870 0.90 -0.89 0.04 
do in science class? 
How often do you talk to 
your friends about what 1,029 072 870 064 -231* O11 | 1,029 0.72 870 0.63 -2.96%* 0.14 
you do in science class? 


* p < 0.05, two-tailed; ** p < 0.01, two-tailed. A negative g indicates that Phase 2 has a higher score relative to Phase 1. 


Table 34. Results of Chi-Square Test: Comparison of North Carolina Elementary Cohort 
Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 
Fall 2011 


Spring 2014 
(X2(2) = 3.50, p = 0.174, Cramer's V = 0.04, g = 0.09) (X2(2) = 1.50, p = 0.473, Cramer's V = 0.03, g = 0.06) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % Sf. n % Sf. n % Sf. n % Sf. 
Yes 493 47.0 0.9 379 42.9 -1.0 502 47.8 -0.4 439 49.7 0.4 
Maybe 462 44.0 -0.6 411 46.6 0.6 476 45.3 0.2 394 446 = -0.2 
No 95 9.0 -0.7 83 10.5 0.8 72 6.9 0.7 50 5.7 -0.8 
Would you like to be a scientist when you are older? 
Fall 2011 Spring 2014 
(X2(2) = 10.52, p = 0.005*, Cramer's V =0.07, g = 0.15) (X2(2) = 1.48, p = 0.478, Cramer's V = 0.03, g = 0.06) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI. n % Sf. n % Sf. n % SI. 
Yes 219 21.1 17 139 16.0 -19 82 79 0.5 60 6.9 -0.6 
Maybe 273 26.3 -14 271 31.3 15 303 29.2 0.4 240 27.7 -0.5 
No 545 52.6 0.0 457 52.7 0.0 652 62.9 -0.5 567 65.4 0.5 


Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 
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Middle School Cohort Results (Figure 8, page 79): Between Group Comparisons 
=» “I! like science” 


As shown in Table 35, the Mann-Whitney U test results revealed that Phase 2 students had higher scores 
(i.e., a higher level of agreement) than Phase 1 students in both Fall 2011 and Spring 2014, but only the 
difference in Spring 2014 was statistically significant (Z = 2.28, p = 0.023, g = -0.13). However, neither of 
associated effect sizes was substantively important. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 35, the Mann-Whitney U test results revealed that Phase 2 students had higher scores 
(i.e., a higher frequency) than Phase 1 students in both Fall 2011 and Spring 2014, but only the difference 
in Spring 2014 was statistically significant (Z = 3.65, p < 0.001, g = -0.20). However, neither of associated 
effect sizes was substantively important. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 35, Phase 2 students had slightly higher scores (i.e., a higher frequency) than Phase 1 
students in both Fall 2011 and Spring 2014. However, the differences were neither statistically significant 
nor substantively important. 


=" “Do you think science will be useful when you are older?” 


As shown in the top section of Table 36, the chi-square test results revealed that the level of agreement 
between Phase 1 and Phase 2 students for this question was not statistically significant in Fall 2011, 
whereas the difference was statistically significant in Spring 2014 (y7(2) = 14.93, p = 0.001, g = 0.20). 
However, neither difference was substantively important. Examination of the cell standardized residuals 
(s.r.) showed that a lower than expected “Yes” response by Phase 1 students was a major contributor to 
the overall statistically significant relationship between Phase and students’ response in Spring 2014. 
Note that unlike the outcomes for the Elementary Cohort, the largest percentage of both Phase 1 and 
Phase 2 students responded “Yes” in Fall 2011, but “Maybe” in Spring 2014. 


= “Would you like to be a scientist when you are older?” 


As shown in the bottom section of Table 36, the chi-square test results revealed that the level of 
agreement between Phase 1 and Phase 2 students for this question was neither statistically significant 
nor substantively important in either Fall 2011 or Spring 2014. Like the outcomes for the Elementary 
Cohort, the largest percentage of Phase 1 and Phase 2 students at both time points responded “No”, with 
the percentage choosing “No” being higher in Spring 2014. 
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Table 35. Results of Mann-Whitney U Test: Comparison of North Carolina Middle School Cohort 


Phase 1 and Phase 2 Students’ Responses 
Fall 2011 Spring 2014 


Phase 1 Phase 2 Phase 1 Phase 2 
N Mean N Mean Z g N Mean N Mean Z g 

| like science. 510 145 899 1.47 0.63 -0.04 | 510 1.15 899 1.22 2.28* — -0.13 
How often do you talk to 

your family about what you 513 0.95 928 0.98 0.85 -0.05 | 513 0.54 928 0.65 3.65*** -0.20 
do in science class? 

How often do you talk to 

your friends about what 510 0.74 923 0.78 0.66 -0.04 | 510 0.64 923 0.70 1.72 -0.10 
you do in science class? 


* p< 0.05, two-tailed; *** p < 0.001, two-tailed. A negative g indicates that Phase 2 has a higher score relative to Phase 1. 


Table 36. Results of Chi-Square Test: Comparison of North Carolina Middle School Cohort 
Phase 1 and Phase 2 Students’ Responses 


Do you think science will be useful to you when you are older? 


Fall 2011 Spring 2014 
(X2(2) = 4.98, p = 0.083, Cramer's V = 0.06, g = 0.12) (X2(2) = 14.93, p = 0.001*, Cramer's V = 0.10, g = 0.20) 
Phase 1 Phase 2 Phase 1 Phase 2 
n % SI n % SI n % SI n % SL 
Yes 281 54.3 -0.4 522 56.2 0.3 187 36.1 -2.2 425 45.7 1.6 
Maybe 227 43.8 0.9 372 40.0 -0.6 269 51.9 1.2 429 46.2 8 -0.9 
No 10 1.9 -1.5 35 3.8 11 62 12.0 1.9 75 8.1 -1.4 


Would you like to be a scientist when you are older? 


Fall 2011 Spring 2014 
(X2(2) = 2.80, p = 0.247, Cramer's V = 0.04, g = 0.09) (X2(2) = 1.37, p = 0.505, Cramer's V = 0.03, g = 0.06) 
Phase 1 Phase 2 Phase 1 Phase 2 

n % Sh n % SI. n % Sf. n % Sf. 
Yes 47 9.3 13 63 6.8 -0.9 20 3.9 0.4 32 3.5 -0.3 
Maybe 158 31.1 -0.4 301 32.6 0.3 112 22.1 -0.8 228 24.7 0.6 
No 303 59.6 -0.2 560 60.6 0.1 376 74.0 0.4 664 71.9 = -0.3 
Note. s.r. = standardized residual. 
*Statistically significant at p < 0.05 
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North Carolina Results: 
Comparison of Student Attitude Toward Science from Fall 2011 to Spring 2014 


The results of the Wilcoxon Signed-rank test for the Elementary Cohort and Middle School Cohort are 
summarized in Table 37 and Table 39, respectively. The results of the marginal homogeneity test for the 
Elementary Cohort and Middle School Cohort are summarized in Table 38 and Table 40, respectively. 


Elementary Cohort Results (Figure 7, page 77): Within Group Comparisons 
=» “like science” 


As shown in Table 37, both Phase 1 and Phase 2 students had higher scores (i.e., a higher level of 
agreement) in Fall 2011 than in Spring 2014. The Wilcoxon Signed-rank test suggested that the within- 
group differences for Phase 1 students were not statistically significant. However, Phase 2 students had 
statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 2.40, p = 0.017, r= -0.06). 
The associated effects for both groups were negligible. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 37, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores (i.e., a higher frequency) in Fall 2011 than in Spring 2014 (Z = 
2.52, p = 0.012, r= -0.06 and Z = 2.77, p = 0.006, r = -0.07, respectively). The associated effects for both 
groups were negligible. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 37, both Phase 1 and Phase 2 students had almost exactly the same scores in both 
Fall 2011 and Spring 2014. The Wilcoxon Signed-rank test suggested that the within-group differences for 
both Phase 1 and Phase 2 students were not statistically significant, and the associated effects for both 
groups were essentially zero. 


* “Do you think science will be useful when you are older?” 


As shown in Table 38, the marginal homogeneity tests showed that there was a statistically significant 
difference in the level of agreement for this question within the Phase 2 student group from Fall 2011 to 
Spring 2014 (MH = -4.10, p < 0.001), with the level of agreement being higher in Spring 2014, whereas 
the difference within the Phase 1 student group, which also had a higher level of agreement in Spring 
2014, was not statistically significant. 


» “Would you like to be a scientist when you are older?” 


As shown in Table 38, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both the Phase 1 and Phase 2 groups from 
Fall 2011 to Spring 2014 (MH = 7.97, p < 0.001 and MH = 7.07, p < 0.001, respectively), with the level of 
agreement being higher in Fall 2011 for both groups. 
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Table 37. Results of Wilcoxon Signed-Rank Test: Comparison of North Carolina Elementary Cohort 


Students’ Responses within the Phase 1 and Phase 2 Groups 
Phase 1 Phase 2 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 
N Mean N Mean Z r N Mean N Mean Z r 
| like science. 1,029 153 1029 151 0.76 -0.02 | 871 1.55 871 1.48 2.40* -0.06 


How often do you talk to 
your family about what 1049 099 1,049 092 252* -0.06 | 870 0.97 870 0.90 2.77 ~~ -0.07 
you do in science class? 

How often do you talk to 
your friends about what 1,029 072 1,029 0.72 -0.02 0.00 | 870 0.64 870 0.63 0.28 -0.01 
you do in science class? 
* p< 0.05, two-tailed; ** p < 0.01, two-tailed. A negative r indicates that Fall 2011 has a higher score relative to Spring 2014. 


Table 38. Results of Marginal Homogeneity Test: Comparison of the Distributions of North Carolina Elementary Cohort 
Students’ Responses within the Phase 1 and Phase 2 Groups 


Marginal Homogeneity Test 


N Standard MH Statistic 
Phase 1 1,050 -1.20 0.230 
Do you think science will be useful to you when you are older? 
Phase 2 883 -4,10 < 0.001* 
Phase 1 1,037 7.97 < 0.001* 
Would you like to be a scientist when you are older? 
Phase 2 867 7.07 < 0.001* 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative to 
Fall 2011. 
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Middle School Cohort Results (Figure 8, page 79): Within Group Comparisons 
=» “I like science” 


As shown in Table 39, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 9.17, p < 0.001, r= -0.29 
and Z = 10.27, p < 0.001, r = -0.24, respectively). The associated effects for both groups were small. 


= “How often do you talk to your family about what you do in science class?” 


As shown in Table 39, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 11.14, p < 0.001, r=- 
0.35 and Z = 12.44, p < 0.001, r = -0.29, respectively). The associated effect for the Phase 1 within-group 
difference was medium in magnitude while the effect for the Phase 2 within-group difference was small in 
magnitude. 


= “How often do you talk to your friends about what you do in science class?” 


As shown in Table 39, the Wilcoxon Signed-rank test suggested that both Phase 1 and Phase 2 students 
had statistically significantly higher scores in Fall 2011 than in Spring 2014 (Z = 2.87, p = 0.004, r = -0.09 
and Z = 2.86, p = 0.004, r = -0.07, respectively). However, the associated effects for both groups were 
negligible. 


*" “Do you think science will be useful when you are older?” 


As shown in Table 40, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both the Phase 1 and Phase 2 groups from 
Fall 2011 to Spring 2014 (MH = 8.16, p < 0.001 and MH = 6.03, p < 0.001, respectively), with the level of 
agreement being higher in Fall 2011 for both groups. 


=» “Would you like to be a scientist when you are older?” 


As shown in Table 40, the marginal homogeneity tests showed that there were statistically significant 
differences in the level of agreement for this question within both the Phase 1 and Phase 2 groups from 
Fall 2011 to Spring 2014 (MH = 6.54, p < 0.001 and MH = 6.41, p < 0.001, respectively), with the level of 
agreement being higher in Fall 2011 for both groups. 
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Table 39. Results of Wilcoxon Signed-Rank Test: Comparison of North Carolina Middle School Cohort 


Students’ Responses within the Phase 1 and Phase 2 Groups 
Phase 1 Phase 2 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 
N Mean N Mean Zz r N Mean N Mean Z r 
| like science. 510 1.45 510 1.15 9.17*** -0.29 | 899 1.47 899 1.22 10.27** = -0.24 


How often do you talk to 
your family about what 513 0.95 513 054 11.14** -0.35 | 928 0.98 928 0.65 12.44** -0.29 
you do in science class? 

How often do you talk to 
your friends about what 510 074 510 0.64 2.87" -0.09 | 923 0.78 923 0.70 2.86*%* — -0.07 
you do in science class? 
** p< 0.01, two-tailed; *** p < 0.001, two-tailed. A negative r indicates that Fall 2011 has a higher score relative to Spring 2014. 


Table 40. Results of Marginal Homogeneity Test: Comparison of the Distributions of North Carolina Middle School Cohort 
Students’ Responses within the Phase 1 and Phase 2 Groups 


Marginal Homogeneity Test 


N Standard MH Statistic 
Phase 1 518 8.16 < 0.001* 
Do you think science will be useful to you when you are older? 
Phase 2 929 6.03 < 0.001* 
Phase 1 508 6.54 < 0.001* 
Would you like to be a scientist when you are older? 
Phase 2 924 6.41 < 0.001* 


*Statistically significant at p < 0.05. For the Standard MH Statistic, a negative value indicates that the Spring 2014 has a higher score relative to 
Fall 2011. 
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Figure 7. Student Attitude Survey, North Carolina: Fall 2011 and Spring 2014: Elementary Cohort Question Responses by Group 


I like Science ... 
Not at all A little A lot 
Phase 4 33.2% 
Fall 2011 


Not at all A little A lot 
Phase 1 EE 39.8% 
Spring 2014 


Phase 2 EE 42.3% 


How often do you talk to your family about what you do in science class? 


Never or almost never Some Alot 
Phase 1 49.5% 
Fall 2011 


Never or almost never Some Alot 
Phase 1 61.5% 
Spring 2014 


Phase 2 64.5% 
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How often do you talk to your friends about what you do in science class? 


Never or almost never Some Alot 
Phase 36.1% 
Fall 2011 


Phase 2 37.2% 


Never or almost never Some A lot 
Phase 1 50.1% 
Spring 2014 
Phase 2 44.6% 47.6% 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase 1 EX 44.0% 
Fall 2011 


Phase 2 UES? 46.6% 


No Maybe Yes 
Phase 1 [EZ 45.3% 
Spring 2014 


Phase 2 44.0% 


Would you like to be a scientist when you are older? 


No Maybe Yes 
Phase 1 26.3% 
Fall 2011 


Phase 2 52.7% 31.3% 


No Maybe Yes 
Phase 1 29.2% 
Spring 2014 
Phase 2 65.4% 27.7% 
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Figure 8. Student Attitude Survey, North Carolina: Fall 2011 and Spring 2014: Middle School Cohort Question Responses by Group 


| like Science... 

Not at all A little A lot 

Phase 1 [E38 45.1% 50.2% 
Fall 2011 

Phase 2 FW 43.3% 52.1% 

Not at all A little Alot 
Phase 1 EUS 63.7% 

Spring 2014 
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How often do you talk to your family about what you do in science class? 


Never or almost never Some A lot 
Phase 1 61.0% 
Fall 2011 


Phase 2 52.6% 


Never or almost never Some Alot 
Phase 1 47.6% 5% 
Spring 2014 


Phase 2 56.3% 
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How often do you talk to your friends about what you do in science class? 


Never or almost never Some A lot 

Phase 1 54.5% 
Fall 2011 

Phase 2 37.5% 47.2% 

Never or almost never Some Alot 

Phase 47.1% 
Spring 2014 

Phase 2 39.9% 50.5% 


Do you think Science will be useful to you when you get older? 


No Maybe Yes 
Phase 1 E 43.8% 
Fall 2011 


Phase 2 [Ey 40.0% 


No Maybe Yes 
Phase 1 51.9% 
Spring 2014 


Phase 2 EET? 46.2% 


Would you like to be a scientist when you are older? 


No Maybe Yes 
Phase 1 31.1% 
Fall 2011 
Phase 2 60.6% 32.6% 
No Maybe Yes 
Phase 1 22.1% 3B 
Spring 2014 
Phase 2 71.9% 24.7% % 
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Overall Summary of North Carolina PASS Student Attitudes Results 


Between-group difference 


For both the Elementary and Middle School cohorts in the North Carolina region, while there were some 
statistically significant findings, particularly in Spring 2014, none of the five items examined exhibited 
meaningful differences between Phase 1 and Phase 2 students in either Fall 2011 or Spring 2014. In 
other words, Phase 1 and Phase 2 students essentially had similar attitudes toward science in both Fall 
2011 and Spring 2014. 


Within-group difference 


For the three Likert-scale items (‘I like science”, “How often do you talk to your family about what you do 
in science class?”, and “How often do you talk to your friends about what you do in science class?”), 
Phase 1 and Phase 2 students in both the Elementary Cohort and the Middle School Cohort liked science 
more or were more likely to talk to their friends or families about science in Fall 2011 compared to Spring 
2014, except for the Phase 1 Elementary Cohort on the item “How often do you talk to your friends about 
what you do in science class?”, where the outcomes were essentially the same at both time points. For 
the Elementary Cohort, the only within group statistically significant differences were found for both 
Phase 1 and Phase 2 students on the item “How often do you talk to your family about what you do in 
science class?” and for the Phase 2 students on the item “I like science”. However, the magnitudes of the 
differences were all negligible. In contrast, the attitudinal changes from Fall 2011 to Spring 2014 within 
both groups in the Middle School Cohort were statistically significant for all three questions. However, 
only the magnitude of the change within the Phase 1 group on the question “How often do you talk to your 
family about what you do in science class?” was neither Negligible nor Small, and reached a Medium 
level. 


For the two nominal scale items (“Do you think science will be useful to you when you are older?” and 
“Would you like to be a scientist when you are older?”), there were statistically significant differences from 
Fall 2011 to Spring 2014 in the level of agreement within both groups for both the Elementary Cohort and 
Middle School Cohort except for the responses of Phase 1 students in the Elementary Cohort to the 
question “Do you think science will be useful to you when you are older?”. In addition, only one of the 
statistically significant differences, for the Phase 2 Elementary Cohort on the question “Do you think 
science will be useful to you when you are older?” had a higher percentage of agreement in Spring 2014 
compared to Fall 2011. Phase 1 students also had a higher level of agreement in Spring 2014 on this 
question, but the difference was not statistically significant. All of the remaining statistically significant 
differences demonstrated higher levels of agreement in Fall 2011. However, since no effect size measure 
for the marginal homogeneity test was available, we do not know whether these statistically significant 
differences were substantively important. 
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Appendix A 


Responses for Fall 2011 & Spring 2014 
Student Attitudes Toward Science 
Survey Items 
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Responses for Fall 2011 & Spring 2014 Student Attitudes Toward Science Survey Items: All Regions 


Table A - 1. PASS, HISD, North Carolina, New Mexico, Fall 2011 & Spring 2014: Elementary Cohort Student Attitude Survey Responses 
My grades in science are usually: 


My grades in math are usually: 


My grades in reading are usually: 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Grades Phase1 Phase 2 Phasel | Phase2 Phase1 Phase 2 Phase1 Phase2 Phase1 | Phase 2 Phase1 Phase 2 
A's 31.9 35.7 39.1 36.1 38.0 43.1 38.3 42.2 36.5 40.6 35.8 34.8 
B's 27.6 28.0 40.8 41.2 25.2 23.3 37.1 36.3 23.7 22.0 38.9 39.8 
C's 8.6 8.5 14.6 16.6 9.7 8.3 15.3 13.5 11.7 9.4 16.7 16.1 
D's 1.2 2.0 3.2 3.9 2.2 2.2 5.4 47 3.0 3.9 5.3 5.4 
F's 0.7 0.7 1.0 1.0 1.6 18 24 2.1 16 2.1 18 2.5 
E's 11 0.8 0.0 0.0 1.0 0.6 0.0 0.0 16 14 0.0 0.0 
S's 17.7 13.6 0.0 0.0 12.5 9.5 0.0 0.0 11.5 8.5 0.0 0.0 
N's 43 43 0.0 0.0 2.5 3.3 0.0 0.0 2.6 3.5 0.0 0.0 
Multiple responses 5.6 4.3 0.0 0.0 45 3.0 0.0 0.0 4.3 2.5 0.0 0.0 
Missing 1.2 21 13 11 27 49 1.6 11 3.5 6.1 15 14 
Fall 2011 Spring 2014 
| like science... Phase1 Phase2 Phase 1 Phase 2 
Alot 65.1 63.8 56.6 52.6 
Alittle 27.2 27.3 37.6 41.5 
Not at all 46 5.5 4.0 4.0 
Multiple responses 0.0 0.2 0.0 0.0 
Missing 3.2 3.2 18 1.9 
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Table A - 1, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
Fall 2011 ia Spring 2014 Fall 2011 Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase2 | Phase Phase 2 
Yes 50.6 47.9 51.2 49.8 23.9 21.2 9.7 9.1 
Maybe 40.3 41.6 42.5 45.1 30.8 31.7 34.5 32.5 
No 6.4 7.8 5.7 47 41.8 43.4 94.2 56.9 
Multiple responses 1.3 0.7 0.0 0.0 0.4 0.2 0.0 0.0 
Missing 15 2.0 0.6 0.4 3.1 3.5 16 15 
How often do you talk to your family about what you doin How often do you talk to your friends about what you do 
science class? in science class? 
Frequency of Science Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Discussion Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
A lot 26.5 25.4 147 12.2 18.5 16.7 11.6 8.4 
Some 47.5 45.5 61.2 64.0 35.7 36.7 48.4 46.9 
Never or almost never 22.1 25.3 23.1 23.0 40.8 42.1 38.6 43.6 
Multiple responses 0.3 0.6 0.0 0.0 0.4 0.4 0.0 0.0 
Missing 3.6 3.3 0.9 0.8 4.6 4.1 13 1.0 
How much effort did you pee! Qed 
put into this test? Phase1 Phase 2 Phase1 Phase 2 

| tried really hard 84.3 84.8 85.3 82.8 
| tried a little 8.4 8.5 5.4 43 
| did not try at all 0.4 1.0 0.3 0.3 
Multiple responses 0.1 0.0 0.0 0.0 
Missing 6.8 5.6 8.9 12.6 
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Table A - 1, continued 
How often do you do the 


Answer questions on worksheets 


Read a science textbook 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 31.6 32.0 34.3 35.0 19.0 21.0 18.0 17.6 
Every week 37.2 36.8 43.1 46.2 26.3 26.2 34.8 32.2 
Every month 9.0 11.7 12.4 11.0 11.1 10.9 20.1 19.7 
Every year 42 44 2.8 15 5.7 2.8 6.1 4.1 
Never 12.1 10.4 5.2 4.0 32.1 34.2 18.7 23.9 
Multiple responses 2.2 19 0.0 0.0 13 1.6 0.0 0.0 
Missing 3.8 2.8 2.2 2.3 45 3.3 24 2.6 


How often do you do the 


Work on experiments with a partner or group 


Work on a project or written report 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 18.6 19.0 27.9 13.1 13.2 13.5 6.3 5.0 
Every week 36.6 35.8 45.9 37.6 23.1 21.6 20.7 15.6 
Every month 15.2 17.4 18.5 34.5 22.9 21.0 45.5 44.4 
Every year 6.3 5.4 2.0 5.6 10.5 12.3 12.5 13.7 
Never 16.3 17.0 2.6 7.0 23.2 25.3 12.0 18.6 
Multiple responses 2.0 1.4 0.0 0.0 0.6 0.8 0.0 0.0 
Missing 5.1 40 3.1 2.1 6.5 54 2.9 27 


Use a science notebook 


How often do you do the 


following activities in Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 43.0 34.8 68.2 55.0 
Every week 29.3 30.2 22.7 21.8 
Every month 6.8 7.6 4.1 go. 
Every year 4.3 4.0 1.4 27 
Never 10.8 18.4 15 9.3 
Multiple responses 0.4 0.4 0.0 0.0 
Missing 5.3 4.7 2.1 2.2 
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Table A - 2. PASS, North Carolina, New Mexico, Fall 2011 & Spring 2014: Middle School Cohort Student Attitude Survey Responses 


My grades in science are usually: My grades in math are usually: My grades in reading are usually: 
Fall 2011 Spring 2014 Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Grades Phase1 Phase 2 Phase1 | Phase2 Phase1 Phase 2 Phase1 Ey Phase1 | Phase 2 Phase1 Phase 2 

A's 40.9 37.1 31.6 30.0 45.6 39.3 28.2 31.5 43.4 40.2 43.5 38.1 
B's 37.8 41.4 32.1 34.9 31.2 38.7 35.6 41.6 33.9 36.9 35.0 40.0 
C's 13.3 12.7 24.2 22.5 13.2 12.1 21.7 17.9 13.3 12.0 14.1 15.6 
D's 2.6 3.6 7.3 78 3.1 2.3 8.6 6.1 2.9 3.1 3.9 4.2 
F's 0.9 0.2 3.1 3.6 18 0.7 3.6 19 1.2 0.5 12 0.9 
E's 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
S's 0.1 0.1 0.0 0.0 0.1 0.0 0.0 0.0 0.2 0.0 0.0 0.0 
N's 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Multiple responses 3.9 4.2 0.0 0.0 3.4 4.1 0.0 0.0 3.2 3.7 0.0 0.0 
Missing 0.5 0.7 17 11 1.6 2.7 2.3 1.0 18 3.5 2.2 14 


—— Fall204. “(i iss—~=—<—~*tsé«<Sizpringg 2024—~C~;~*=*Y 
| like science... Phase1 Phase2 Phase 1 Phase 2 
Alot 51.1 48.8 28.2 29.6 
A little 43.1 43.8 58.2 59.8 
Not at all 3.7 3.8 12.0 9.2 
Multiple responses 0.1 0.0 0.0 0.0 
Missing 2.0 3.6 16 14 


Summative Report Section 5: Student Attitudes 77 


Table A - 2, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
2 
Fall 2011 = Spring 2014 Fall 2011 | Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase2 | Phase Phase 2 
Yes 53.8 53.7 35.6 44.1 8.9 7.2 46 3.9 
Maybe 41.8 40.8 49.8 47.3 34.2 31.2 22.6 24.4 
No 2.6 3.5 13.8 8.1 55.0 59.3 70.3 70.5 
Multiple responses 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 15 2.1 0.9 0.5 18 2.2 2.5 1.2 


How often do you talk to your family about what you doin How often do you talk to your friends about what you do 


science class? in science class? 
Frequency of Science Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Discussion Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Alot 18.9 17.1 5.4 49 11.2 13.6 78 8.7 
Some 57.8 60.3 44.9 04.8 50.2 47.3 45.3 49.0 
Never or almost never 21.6 20.0 48.3 39.5 36.4 36.5 45.6 40.9 
Multiple responses 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 17 2.4 14 0.8 2.2 25 13 14 
How much effort did you neat Qed 
put into this test? Phase1 Phase 2 Phase1 Phase 2 
| tried really hard 80.8 80.5 65.3 68.9 
| tried a little 15.2 12.5 20.7 18.2 
| did not try at all 0.3 0.4 es 1.0 
Multiple responses 0.1 0.0 0.0 0.0 
Missing 3.6 6.6 12.3 11.9 
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Table A - 2, continued 
How often do you do the 


Answer questions on worksheets 


Read a science textbook 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 43.8 31.1 32.8 24.7 35.0 29.7 24.7 17.3 
Every week 45.0 53.8 51.6 58.2 45.9 38.2 49.5 29.8 
Every month 6.3 71 9.8 12.1 9.1 16.2 16.7 31.7 
Every year 0.9 2.0 0.5 0.6 0.8 15 3.1 4.8 
Never 18 24 0.7 3.1 7.3 9.9 18 14.9 
Multiple responses 0.5 0.3 0.0 0.0 0.2 0.6 0.0 0.0 
Missing 17 3.4 46 1.3 1.6 3.9 42 15 


How often do you do the 


Work on experiments with a partner or group 


Work on a project or written report 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phasel Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 13.8 13.0 9.6 6.0 8.4 (Al 2.3 48 
Every week 53.6 48.1 44.9 35.8 24.5 19.9 13.7 21.3 
Every month 25.4 24.8 32.4 49.6 43.0 45.2 52.8 61.6 
Every year 0.9 1.4 4.3 3.0 5.6 6.6 19.0 7.4 
Never 3.5 8.2 3.6 3.8 15.8 15.7 78 2.9 
Multiple responses 0.4 0.9 0.0 0.0 0.3 0.4 0.0 0.0 
Missing 24 3.6 5.2 1.9 2.3 45 46 2.0 


How often do you do the 


Use a science notebook 


following activities in Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 62.9 69.6 61.5 67.3 
Every week 22.2 11.4 15.4 16.4 
Every month 44 3.3 5.7 5.7 
Every year 1.2 1.9 1.8 1.6 
Never 7.8 10.1 11.7 77 
Multiple responses 14 0.3 0.0 0.0 
Missing 62.9 3.4 4.0 1.3 
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Responses for Fall 2011 & Spring 2014 Student Attitudes Toward Science Survey Items: HISD 


Table A - 3. PASS, HISD, Fall 2011 & Sp 


My grades in science are usually: 


ring 2014: Elementary Cohort Student Attitude Survey Responses 


My grades in math are usually: 


My grades in reading are usually: 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Grades Phase1 Phase 2 Phasel | Phase2 Phase1 Phase 2 Phase1 Phase2 Phase1 | Phase 2 Phase1 Phase 2 
A's 35.3 36.0 25.7 21.2 38.5 37.8 30.2 31.4 32.0 34.6 23.4 21.7 
B's 34.5 34.1 46.1 45.9 28.7 29.4 39.4 41.6 30.4 27.7 38.4 40.7 
C's 13.1 11.8 20.8 25.0 13.7 12.7 18.9 16.4 17.3 13.9 26.0 23.5 
D's 2.4 3.0 5.4 5.2 3.2 3.6 7.3 7.3 45 5.8 8.0 (A 
F's 0.5 0.9 13 1.2 2.1 2.2 3.4 18 2.0 24 3.5 48 
E's 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.1 0.4 0.0 0.0 
S's 13 0.4 0.0 0.0 0.9 0.7 0.0 0.0 0.9 0.2 0.0 0.0 
N's 1.3 0.9 0.0 0.0 0.7 0.2 0.0 0.0 0.8 0.6 0.0 0.0 
Multiple responses 10.7 9.9 0.0 0.0 9.6 719 0.0 0.0 8.9 73 0.0 0.0 
Missing 0.8 3.0 0.9 15 2.5 5.2 0.9 15 2.9 71 0.6 15 
Fall 2011 Spring 2014 
| like science... Phase1 Phase2 Phase 1 Phase 2 
Alot 68.7 69.5 58.8 55.9 
Alittle 23.7 23.0 35.4 38.9 
Not at all 3.5 3.0 3.4 2.5 
Multiple responses 0.0 0.2 0.0 0.0 
Missing 4.1 4.3 24 2.7 
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Table A - 3, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
Fall 2011 ia Spring 2014 Fall 2011 Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase2 | Phase Phase 2 
Yes 50.8 53.2 53.3 48.6 30.0 29.4 9.6 12.2 
Maybe 40.1 38.4 41.1 48.6 33.2 35.6 41.1 39.6 
No 5.1 5.4 5.0 2.2 32.5 30.1 47.3 46.7 
Multiple responses 0.5 0.7 0.0 0.0 0.4 0.6 0.0 0.0 
Missing 3.5 2.2 0.6 0.7 3.9 43 2.0 15 
How often do you talk to your family about what you doin How often do you talk to your friends about what you do 
science class? in science class? 
Frequency of Science Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Discussion Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
A lot 31.6 30.5 14.4 11.4 22.9 24.5 14.0 9.7 
Some 44.0 44.9 62.6 65.4 38.5 40.4 48.9 51:3 
Never or almost never 19.5 19.9 21.7 22.4 32.8 29.0 36.2 38.2 
Multiple responses 0.4 0.6 0.0 0.0 0.1 0.7 0.0 0.0 
Missing 45 4.1 14 0.8 5.6 5.2 0.9 0.8 
How much effort did you neat Qed 
put into this test? Phase1 Phase 2 Phase1 Phase 2 

| tried really hard 81.2 78.1 87.0 82.8 
| tried a little 9.6 12.5 6.9 6.2 
| did not try at all 0.1 13 0.1 0.2 
Multiple responses 0.1 0.0 0.0 0.0 
Missing 8.9 8.1 6.0 10.9 


Summative Report Section 5: Student Attitudes 


81 


Table A - 3, continued 
How often do you do the 


Answer questions on worksheets 


Read a science textbook 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 31.3 34.6 52.9 50.1 24.1 27.0 20.0 17.0 
Every week 43.5 32.0 35.7 39.6 32.8 38.4 32.8 35.4 
Every month 7.9 18.2 6.0 48 16.1 12.7 18.8 15.5 
Every year 2.8 2.1 0.6 1.0 45 2.8 5.8 45 
Never 71 71 2.8 27 15.6 12.7 20.5 25.4 
Multiple responses 13 2.2 0.0 0.0 0.7 17 0.0 0.0 
Missing 6.1 3.7 2.0 18 6.1 47 2.1 2.2 


How often do you do the 


Work on experiments with a partner or group 


Work on a project or written report 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 20.4 25.7 18.9 21.0 13.2 17.8 6.1 7.8 
Every week 45.1 45.9 49.6 48.6 25.2 22.8 21.0 21.9 
Every month 15.1 12.4 23.8 21.5 28.5 18.2 46.1 41.9 
Every year 45 3.7 14 3.0 8.7 17.0 11.1 9.7 
Never 6.1 6.0 41 4.0 16.1 15.9 13.1 16.5 
Multiple responses 1.6 0.7 0.0 0.0 0.5 1.3 0.0 0.0 
Missing 7.2 5.6 2.3 18 77 6.9 2.5 2.2 
How often do you do the Use a science notebook 
following activities in Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 57.3 48.7 73.0 69.1 
Every week 26.1 34.6 18.9 24.9 
Every month 43 3.0 3.9 3.7 
Every year 2.9 3.6 11 0.7 
Never 2.5 3.7 11 0.3 
Multiple responses 0.0 0.4 0.0 0.0 
Missing 6.8 6.0 2.0 1.3 
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Table A - 4. PASS, HISD, Spring 2012 & Spring 2014: Middle School Cohort Student Attitude Survey Responses 


My grades in science are usually: My grades in math are usually: My grades in reading are usually: 
Spring 2012 Spring 2014 Spring 2012 Spring 2014 Spring 2012 Spring 2014 
Grades Phase1 Phase 2 Phase1 | Phase2 Phase1 Phase 2 Phase1 Phase2 Phase1 | Phase 2 Phase1 Phase 2 
A's 15.7 15.1 16.8 19.8 28.5 27.7 27.9 24.0 24.4 17.6 27.9 24.0 
B's 57.0 42.0 38.5 54.5 36.6 35.3 50.8 45.5 41.9 46.2 49.2 50.4 
C's 15.7 25.2 31.3 19.8 20.9 21.0 16.2 19.8 15.7 18.5 17.9 17.4 
D's 1.2 5.0 7.3 17 17 2.5 2.2 5.8 3.5 5.0 2.2 17 
F's 0.0 0.0 5.0 0.0 12 17 17 0.0 2.3 17 11 17 
E's 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
S's 0.0 0.0 0.0 0.0 0.0 0.8 0.0 0.0 0.0 0.8 0.0 0.0 
N's 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Multiple responses 9.3 11.8 0.0 0.0 8.7 10.1 0.0 0.0 8.7 9.2 0.0 0.0 
Missing 1.2 0.8 11 4.1 2.3 0.8 11 5.0 3.5 0.8 17 5.0 


-—- Spring2012—s—“(s=“‘<«~*‘;é*SS zing 2024~C~*=‘*Y'T 
| like science... Phasel Phase2 Phase 1 Phase 2 
Alot 32.6 52.1 20.7 38.8 
A little 56.4 41.2 63.1 52.9 
Not at all 8.1 17 15.1 2.5 
Multiple responses 0.0 0.0 0.0 0.0 
Missing 2.9 5.0 11 5.8 
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Table A - 4, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
2 
Spring 2012 = Spring 2014 Spring 2012 | Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase2 | Phase Phase 2 
Yes 54.7 62.2 34.6 41.3 AT 8.4 2.8 9.9 
Maybe 39.5 33.6 48.6 49.6 31.4 43.7 20.1 24.8 
No 47 0.8 16.2 5.8 63.4 44.5 15.4 59.5 
Multiple responses 0.0 0.0 0.0 0.0 0.0 0.8 0.0 0.0 
Missing 1.2 3.4 0.6 3.3 0.6 2.5 17 5.8 


How often do you talk to your family about what you doin How often do you talk to your friends about what you do 


science class? in science class? 
Frequency of Science Spring 2012 Spring 2014 Spring 2012 Spring 2014 
Discussion Phase1 Ee W4 Phase1 Ew Phase1 Ee W4 Phase1 Phase 2 
A lot 13.4 15.1 2.8 5.0 10.5 18.5 7.3 8.3 
Some 45.9 60.5 36.9 52.9 43.6 54.6 40.2 52.1 
Never or almost never 39.5 21.0 60.3 38.0 44.2 23.5 52.5 35.5 
Multiple responses 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 1.2 3.4 0.0 4.1 17 3.4 0.0 4.1 


How much effort did you eal ou Qed 
put into this test? Phase1 Phase 2 Phase1 Phase 2 
| tried really hard 73.3 66.4 69.8 777 
| tried a little 14.5 10.1 19.6 16.5 
| did not try at all 2.3 0.0 17 0.0 
Multiple responses 0.0 0.0 0.0 0.0 
Missing 9.9 23.5 8.9 5.8 
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Table A - 4, continued 


How often do you do the Answer questions on worksheets Read a science textbook 
following activities in Spring 2012 Spring 2014 Spring 2012 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 44.8 57.1 62.6 69.4 30.2 26.9 22.9 8.3 
Every week 32.6 25.2 30.7 24.0 50.6 44.5 48.6 30.6 
Every month 15.1 42 2.8 0.8 2.9 11.8 15.1 22.3 
Every year 1.2 2.5 0.6 0.8 1.2 17 2.2 5.8 
Never 47 3.4 2.2 17 12.2 8.4 9.5 29.8 
Multiple responses 0.0 0.8 0.0 0.0 0.0 0.8 0.0 0.0 
Missing 17 6.7 11 3.3 2.9 5.9 17 3.3 
How often do you do the Work on experiments with a partner or group Work on a project or written report 
following activities in Spring 2012 Spring 2014 Spring 2012 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 16.3 31.1 22.3 15.7 7.6 13.4 8.4 5.0 
Every week 40.7 35.3 49.2 44.6 22.1 26.9 17.3 20.7 
Every month 18.6 19.3 19.6 28.9 36.6 30.3 48.6 38.0 
Every year 41 0.8 17 3.3 9.9 3.4 9.5 11.6 
Never 16.3 76 6.1 5.0 20.3 19.3 145 21.5 
Multiple responses 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 
Missing 41 5.9 11 2.5 2.9 6.7 17 3.3 
How often do you do the Use a science notebook 
following activities in Spring 2012 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 58.7 69.7 60.9 63.6 
Every week 25.0 16.8 19.0 19.0 
Every month 41 0.8 7.3 6.6 
Every year 5.8 0.8 17 0.8 
Never 17 5.0 10.1 6.6 
Multiple responses 0.6 0.0 0.0 0.0 
Missing 41 6.7 11 3.3 
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Responses for Fall 2011 & Spring 2014 Student Attitudes Toward Science Survey Items: New Mexico 


Table A - 5. PASS, New Mexico, Fall 2011 & Spring 2014: Elementary Cohort Student Attitude Survey Responses 


My grades in science are usually: My grades in math are usually: My grades in reading are usually: 
Fall 2011 Spring 2014 Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Grades Phase1 Phase 2 Phasel | Phase2 Phase1 Phase 2 Phase1 Phase2 Phase1 | Phase 2 Phase1 Phase 2 
A's 38.2 41.8 50.3 39.7 38.7 40.9 47.0 44.8 43.7 43.9 50.1 46.2 
B's 32.9 30.6 37.6 43.9 27.9 26.6 33.1 32.2 22.6 20.9 32.1 32.6 
C's 10.1 12.8 8.6 11.3 13.9 10.0 11.7 14.6 13.3 11.4 10.6 12.0 
D's 0.7 1.0 2.3 3.1 18 1.9 5.1 47 2.8 3.6 41 49 
F's 0.2 0.2 0.6 0.2 14 1.0 1.6 2.1 11 1.2 14 1.4 
E's 1.2 0.0 0.0 0.0 0.5 0.2 0.0 0.0 0.5 0.5 0.0 0.0 
S's 3.9 3.8 0.0 0.0 5.2 3.1 0.0 0.0 5.3 3.1 0.0 0.0 
N's 9.9 5.0 0.0 0.0 71 5.0 0.0 0.0 6.6 45 0.0 0.0 
Multiple responses 14 0.7 0.0 0.0 11 1.0 0.0 0.0 11 0.2 0.0 0.0 
Missing 14 40 0.6 1.9 2.5 10.5 1.6 16 3.0 10.7 18 2.8 
Fall 2011 Spring 2014 
| like science... Phase1 Phase2 Phase 1 Phase 2 

Alot 15.7 60.6 O71 50.0 

Alittle 19.9 31.4 38.0 43.4 

Not at all 27 3.3 3.7 4.0 

Multiple responses 0.0 0.0 0.0 0.0 

Missing 18 4.8 1.2 2.6 
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Table A - 5, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
2 
Fall 2011 sa Spring 2014 Fall 2011 | Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 

Yes 60.6 52.0 57.5 50.2 24.2 21.9 143 9.9 
Maybe 34.5 38.5 38.2 43.9 37.8 30.2 36.8 34.5 
No 3.2 5.2 14 5.4 34.6 43.5 47.9 53.1 
Multiple responses 0.5 0.2 0.0 0.0 0.4 0.0 0.0 0.0 
Missing 1.2 4.0 0.2 0.5 3.0 45 1.0 2.6 

How often do you talk to your family about what you doin —_ How often do you talk to your friends about what you do 

science class? in science class? 
Frequency of Science Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Discussion Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Alot 24.2 21.4 147 12.7 16.2 15.0 10.2 7.3 
Some 92.9 43.0 59.9 61.0 34.6 32.5 47.4 41.1 
Never or almost never 18.1 30.6 24.9 24.9 43.3 47.0 40.7 49.3 
Multiple responses 0.4 0.2 0.0 0.0 0.2 0.2 0.0 0.0 
Missing 44 48 0.6 14 5.7 5.2 18 2.3 
How much effort did you pee! Qed 
put into this test? Phase1 Phase 2 Phase1 Phase 2 

| tried really hard 83.5 774 79.5 76.5 
| tried a little 9.4 11.2 49 5.4 
| did not try at all 0.2 1.0 1.2 0.5 
Multiple responses 0.0 0.0 0.0 0.0 
Missing 6.9 10.5 14.5 17.6 
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Table A - 5, continued 
How often do you do the 


Answer questions on worksheets 


Read a science textbook 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 22.4 19.0 13.5 25.6 11.9 11.9 10.6 29.6 
Every week 40.0 44.9 41.9 45.5 25.6 23.3 33.3 37.1 
Every month 12.1 11.4 21.3 20.4 11.2 10.9 22.1 20.9 
Every year 5.9 3.1 8.4 0.9 11.7 1.2 11.7 3.8 
Never 13.5 16.6 13.7 54 30.9 47.0 21.1 5.6 
Multiple responses 14 17 0.0 0.0 16 14 0.0 0.0 
Missing 48 3.3 1.2 2.1 “1 4.3 1.2 3.1 


How often do you do the 


Work on experiments with a partner or group 


Work on a project or written report 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 15.8 10.5 36.2 6.3 11.7 10.2 8.8 45 
Every week 32.5 36.8 46.4 28.2 23.8 22.1 23.9 14.1 
Every month 16.7 21.9 10.6 45.5 22.9 25.9 35.8 51.2 
Every year 8.2 48 2.9 9.2 11.5 10.7 16.0 16.2 
Never 17.4 20.9 2.2 8.7 22.2 25.2 13.7 11.7 
Multiple responses 2.7 14 0.0 0.0 0.9 0.2 0.0 0.0 
Missing 6.7 3.8 18 21 6.9 5.7 1.8 2.3 
How often do you do the Use a science notebook 
following activities in Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 30.9 15.0 44.0 21.1 
Every week 36.1 29.5 43.1 23.9 
Every month 9.1 14.0 5.9 21.4 
Every year 3.0 4.0 2.0 47 
Never 13.1 32.8 3.7 26.1 
Multiple responses 0.9 0.0 0.0 0.0 
Missing 6.9 48 1.4 2.8 
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Table A - 6. PASS, New Mexico, Fall 2011 & Spring 2014: Middle School Cohort Student Attitude Survey Responses 


My grades in science are usually: My grades in math are usually: My grades in reading are usually: 
Fall 2011 Spring 2014 Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Grades Phase1 Phase 2 Phase1 | Phase2 Phase1 Phase 2 Phase1 Phase2 Phase1 | Phase 2 Phase1 Phase 2 

A's 48.0 39.2 29.1 39.3 54.2 36.1 25.7 17.9 56.1 42.3 49.0 38.6 
B's 34.6 35.1 32.5 22.8 27.9 38.1 29.9 35.2 28.2 32.0 32.9 36.6 
C's 10.8 17.5 23.9 24.1 10.5 18.6 25.3 30.3 8.3 14.4 11.2 17.9 
D's 2.7 7.2 8.4 10.3 2.5 3.1 10.6 14.5 2.7 5.2 3.8 48 
F's 1.2 0.0 44 2.1 1.2 0.0 5.4 2.1 1.2 1.0 0.8 0.7 
E's 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
S's 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
N's 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Multiple responses 2.2 1.0 0.0 0.0 17 0.0 0.0 0.0 15 0.0 0.0 0.0 
Missing 0.5 0.0 18 14 2.0 4.1 3.2 0.0 2.0 5.2 2.4 14 


Fall 2011 '- Spring20144 
| like science... Phase1 Phase2 Phase 1 Phase 2 
Alot 54.4 40.2 29.9 23.4 
Alittle 41.4 52.6 54.0 62.1 
Not at all 1.2 2.1 14.1 13.1 
Multiple responses 0.2 0.0 0.0 0.0 
Missing 2.7 5.2 2.0 14 
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Table A - 6, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
Fall 2011 aa Spring 2014 Fall 2011 | Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase2 | Phase Phase 2 
Yes 55.4 47.4 34.3 33.1 8.8 41 5.6 5.5 
Maybe 38.7 38.1 48.6 53.8 39.5 36.1 23.7 21.4 
No 3.2 5.2 16.3 11.7 49.0 48.5 68.7 71.0 
Multiple responses 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 2.5 9.3 0.8 1.4 2.7 11.3 2.0 2.1 


How often do you talk to your family about what youdoin How often do you talk to your friends about what you do 


science class? in science class? 
Frequency of Science Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Discussion Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Alot 21.1 20.6 74 7.6 13.7 12.4 6.6 5.5 
Some 57.1 47.4 42.6 45.5 45.8 41.2 43.8 44.1 
Never or almost never 19.1 19.6 49.0 44.8 37.7 34.0 48.8 46.2 
Multiple responses 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 2.7 12.4 1.0 2.1 2.7 12.4 0.8 4.1 
How much effort did you pee! Qed 
put into this test? Phase1 Phase 2 Phase1 Phase 2 
| tried really hard 82.8 73.2 64.1 51.7 
| tried a little 12.7 17.5 22.7 18.6 
| did not try at all 0.5 0.0 2.0 14 
Multiple responses 0.0 0.0 0.0 0.0 
Missing 3.9 9.3 11.2 28.3 
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Table A - 6, continued 


How often do you do the Answer questions on worksheets Read a science textbook 
following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 40.2 33.0 29.3 25.5 29.4 39.2 19.9 56.6 
Every week 47.3 34.0 60.6 51.7 55.9 36.1 58.8 26.9 
Every month 78 10.3 6.0 15.2 10.5 41 15.5 9.0 
Every year 0.2 5.2 0.2 1.4 0.7 0.0 2.0 2.1 
Never 17 6.2 0.2 6.2 15 7.2 0.8 41 
Multiple responses 0.7 0.0 0.0 0.0 0.0 3 0.0 0.0 
Missing 2.0 11.3 3.8 0.0 2.0 10.3 3.4 1.4 
How often do you do the Work on experiments with a partner or group Work on a project or written report 
following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 ie Phase1 Phase 2 
Every day 9.3 14.4 14.9 3.4 8.8 17.5 3.4 6.9 
Every week 47.5 42.3 59.4 38.6 16.9 21.6 10.8 14.5 
Every month 35.0 20.6 19.3 44.8 53.4 32.0 52.6 40.7 
Every year 0.7 3.1 1.2 2.8 74 7.2 18.7 22.8 
Never 3.9 7.2 0.8 9.0 10.5 9.3 10.8 11.7 
Multiple responses 0.2 1.0 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 3.2 11.3 44 1.4 2.9 12.4 3.8 3.4 
How often do you do the Use a science notebook 
following activities in Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 54.9 57.7 59.8 57.2 
Every week 23.3 10.3 14.1 14.5 
Every month 5.6 5.2 3.8 6.9 
Every year 17 6.2 2.8 2.1 
Never 13.0 8.2 15.7 18.6 
Multiple responses 0.0 1.0 0.0 0.0 
Missing 15 11.3 3.8 0.7 
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Responses for Fall 2011 & Spring 2014 Student Attitudes Toward Science Survey Items: North Carolina 


Table A - 7. PASS, North Carolina, Fall 2011 & Sp 


ring 2014: Elementary Cohort Student Attitude Survey Responses 


My grades in science are usually: 


My grades in math are usually: 


My grades in reading are usually: 


Fall 2011 Spring 2014 Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Grades Phase1 Phase 2 Phasel | Phase2 Phase1 Phase 2 Phase1 Phase2 Phase1 | Phase 2 Phase1 Phase 2 
A's 26.5 33.0 43.6 43.7 37.3 46.9 40.2 47.6 35.9 42.4 38.0 37.9 
B's 20.4 23.4 38.4 36.9 21.6 18.4 37.1 34.8 19.9 19.4 42.4 42.4 
C's 49 48 12.9 13.8 5.0 5.2 14.3 11.5 7.2 6.1 12.8 13.1 
D's 0.7 19 2.0 3.6 1.8 17 42 3.2 2.0 2.9 3.9 42 
F's 11 0.7 11 1.3 1.2 1.9 21 2.3 16 2.3 0.9 15 
E's 17 16 0.0 0.0 1.9 1.0 0.0 0.0 3.1 2.3 0.0 0.0 
S's 35.4 25.2 0.0 0.0 23.9 17.2 0.0 0.0 21.6 15.5 0.0 0.0 
N's 3.4 5.9 0.0 0.0 14 42 0.0 0.0 18 48 0.0 0.0 
Multiple responses 44 2.8 0.0 0.0 2.9 11 0.0 0.0 2.9 0.8 0.0 0.0 
Missing 13 0.8 2.0 0.6 2.9 2.3 2.0 0.7 41 3.5 2.0 0.8 
Fall 2011 Spring 2014 
| like science... Phase1 Phase2 Phase 1 Phase 2 
Alot o7.4 62.0 04.7 91.3 
Alittle 33.0 28.0 39.1 42.7 
Not at all 6.3 79 46 49 
Multiple responses 0.0 0.2 0.0 0.0 
Missing 3.3 2.0 17 11 
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Table A - 7, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
Fall 2011 sa Spring 2014 Fall 2011 | Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase2 | Phase Phase 2 

Yes 45.4 43.1 46.8 50.3 19.7 16.4 77 6.5 
Maybe 43.3 44.8 45.5 43.5 25.7 30.2 28.9 27.4 
No 8.8 10.2 6.9 6.1 51.5 50.7 61.9 65.0 
Multiple responses 2.1 0.9 0.0 0.0 0.0 0.1 0.0 0.0 
Missing 0.4 1.0 0.7 0.2 27 27 15 Val 


How often do you talk to your family about what you doin = How often do you talk to your friends about what you do 


science class? in science class? 
Frequency of Science Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Discussion Phasel Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
A lot 24.2 24.2 15.0 12.6 16.8 13.2 10.5 8.3 
Some 47.1 46.8 60.8 64.5 34.3 36.5 48.7 46.9 
Never or almost never 25.9 25.9 23.4 22.3 44.9 47.2 39.4 44.2 
Multiple responses 0.3 0.7 0.0 0.0 0.7 0.3 0.0 0.0 
Missing 2.6 2.3 0.8 0.6 3.4 2.9 14 0.6 
How much effort did you Fall 2011 Spring 2014 
put into this test? Phase1 Phase 2 Phase1 Phase 2 
| tried really hard 86.7 91.7 86.8 85.5 
| tried a little 7.1 5.2 4.6 2.7 
| did not try at all 0.8 0.9 0.1 0.3 
Multiple responses 0.1 0.0 0.0 0.0 
Missing 5.3 2.2 8.5 11.5 
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Table A - 7, continued 
How often do you do the 


Answer questions on worksheets 


Read a science textbook 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 36.4 36.2 30.5 30.1 19.2 21.7 19.9 12.9 
Every week 31.7 36.0 48.9 50.3 22.3 20.7 36.9 28.1 
Every month 8.1 8.3 12.8 10.7 78 9.9 20.0 21.5 
Every year 42 6.2 18 2.1 3.5 3.4 3.7 40 
Never 14.7 9.4 3.1 43 43.6 40.4 16.4 30.8 
Multiple responses 3.1 19 0.0 0.0 1.6 17 0.0 0.0 
Missing 18 2.1 2.8 27 2.0 2.2 3.1 27 
How often do you do the Work on experiments with a partner or group Work on a project or written report 
following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 18.8 19.1 30.6 10.9 13.9 12.6 5.4 3.6 
Every week 33.0 29.7 43.1 34.8 21.5 20.7 19.0 12.1 
Every month 145 18.2 18.3 37.9 19.3 20.5 49.4 43.2 
Every year 6.4 6.6 2.1 5.7 11.1 10.5 12.0 14.9 
Never 22.4 21.3 18 8.3 28.4 30.5 10.5 23.1 
Multiple responses 1.9 1.8 0.0 0.0 0.4 0.7 0.0 0.0 
Missing 2.9 3.2 42 24 5.5 45 3.8 3.2 
How often do you do the Use a science notebook 
following activities in Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 39.6 35.8 75.6 61.1 
Every week 28.1 28.0 16.3 18.6 
Every month 74 7.3 3.4 7.2 
Every year 5.8 4.2 1.4 3.1 
Never 15.1 20.3 0.8 7.6 
Multiple responses 0.5 0.5 0.0 0.0 
Missing 3.4 3.9 2.5 2.5 
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Table A - 8. PASS, North Carolina, Fall 2011 & Spring 2014: Middle School Cohort Student Attitude Survey Responses 


My grades in science are usually: My grades in math are usually: My grades in reading are usually: 
Fall 2011 Spring 2014 Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Grades Phase1 Phase 2 Phase1 | Phase2 Phase1 Phase 2 Phase1 Ey Phase1 | Phase 2 Phase1 Phase 2 

A's 35.9 36.9 33.9 28.7 39.5 39.6 30.4 33.5 34.3 40.0 38.6 38.0 
B's 40.0 42.0 31.7 36.7 33.4 38.7 40.8 42.5 38.0 37.4 37.0 40.4 
C's 15.1. 12.2 24.5 22.3 15.1 11.5 18.5 16.1 16.8 11.8 16.8 15.2 
D's 2.6 3.2 6.3 7.5 3.6 2.3 6.9 4.9 3.1 2.9 4.0 4.1 
F's 0.7 0.2 2.0 3.8 2.3 0.8 2.0 18 1.2 0.5 16 0.9 
E's 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
S's 0.2 0.1 0.0 0.0 0.2 0.0 0.0 0.0 0.3 0.0 0.0 0.0 
N's 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Multiple responses 5.0 4.5 0.0 0.0 4.5 4.5 0.0 0.0 4.5 4.0 0.0 0.0 
Missing 0.5 0.8 1.6 11 14 2.6 14 1.2 17 3.3 2.0 14 


—— Fall204. = s—<—~*stsés«SS pring 2024—~Ct~«=C*Y 
| like science... Phase1 Phase2 Phase 1 Phase 2 
A lot 48.7 49.7 26.6 30.5 
A little 44.4 43.0 62.0 59.5 
Not at all 5.4 3.9 10.1 8.6 
Multiple responses 0.0 0.0 0.0 0.0 
Missing 16 3.4 13 15 
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Table A - 8, continued 


Do you think science will be useful to you when you are Would you like to be a scientist when you are older? 
older? 
Fall 2011 Spring 2013 Fall 2011 | Spring 2014 
Science in my Future Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase2 | Phase Phase 2 
Yes 52.7 54.3 36.8 45.7 9.0 7.5 3.8 3.7 
Maybe 44.0 41.0 50.9 46.4 30.5 30.8 21.6 24.8 
No 2.3 3.3 11.4 76 59.3 60.4 717 70.4 
Multiple responses 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 0.9 1.4 0.9 0.4 1.2 1.4 2.9 11 


How often do you talk to your family about what you doin = How often do you talk to your friends about what you do 


science class? in science class? 
Frequency of Science Fall 2011 Spring 2014 Fall 2011 Spring 2014 
Discussion Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Alot 17.3 16.8 3.6 46 9.4 13.8 8.9 9.1 
Some 58.2 61.6 46.9 96.1 53.2 47.9 46.6 49.7 
Never or almost never 23.4 20.1 47.6 38.8 35.5 36.8 42.8 40.2 
Multiple responses 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 
Missing 1.0 15 18 0.6 19 1.6 18 11 
How much effort did you pee! Qed 
put into this test? Phase1 Phase 2 Phase1 Phase 2 
| tried really hard 79.4 81.2 66.3 71.3 
| tried a little 17.0 12.0 18.8 18.1 
| did not try at all 0.2 0.4 14 1.0 
Multiple responses 0.2 0.0 0.0 0.0 
Missing 3.3 6.4 13.4 9.6 
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Table A - 8, continued 
How often do you do the 


Answer questions on worksheets 


Read a science textbook 


following activities in Fall 2011 Spring 2014 Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 Phase1 Phase 2 
Every day 46.3 30.9 36.1 24.6 39.0 28.8 29.0 11.7 
Every week 43.3 55.7 43.5 59.1 38.8 38.4 41.5 30.2 
Every month 5.2 6.8 13.2 11.6 8.1 17.3 17.8 34.9 
Every year 14 17 0.7 0.5 0.9 17 4.2 5.2 
Never 19 2.1 11 27 11.4 10.1 27 16.4 
Multiple responses 0.3 0.3 0.0 0.0 0.3 0.4 0.0 0.0 
Missing 16 27 5.4 15 14 3.2 4.9 1.6 


How often do you do the 


Work on experiments with a partner or group 


Work on a project or written report 


following activities in Fall 2011 Spring 2014 | Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phasel --Phase2 |  Phasel Phase 2 Phase1 Phase 2 
Every day 17.0 12.9 47 6.3 8.1 6.8 13 45 
Every week 57.9 48.7 31.7 35.4 29.8 19.8 16.3 22.3 
Every month 18.5 25.2 44.4 50.2 35.7 46.5 52.9 64.6 
Every year 1.0 13 G1 3.0 4.3 6.5 19.2 5.2 
Never 3.1 8.3 6.2 3.1 19.6 16.3 54 1.6 
Multiple responses 0.5 0.9 0.0 0.0 0.5 0.4 0.0 0.0 
Missing 19 2.9 6.0 19 19 3.7 5.3 17 


How often do you do the 


Use a science notebook 


following activities in Fall 2011 Spring 2014 
science class? Phase1 Phase 2 Phase1 Phase 2 
Every day 68.6 70.7 63.0 68.8 
Every week 21.5 11.5 16.5 16.7 
Every month 3.5 3.1 7A 5.9 
Every year 0.9 15 0.9 1.6 
Never 4.2 10.3 8.0 6.1 
Multiple responses 0.0 0.2 0.0 0.0 
Missing 1.4 27 4.2 1.4 
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